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Introduction 



In order to determine the effectiveness of the CERLI program 
for training a Specialist In Continuing Education (SCE), we 
needed to find out whether the program had effected behavioral 
changes enabling the participants to function In this new role. 

Testing this hypothesis involved attempts to answer the following 
questions: 



Did t/ie program do what it intended to do? 

Did the behccoior of the participants change? 

If so, in what direction? 

In dealing with these questions, we utilized data derived from 
11 Instruments described in "Analysis”, section I of this report. 
For information about the theoretical basis for and actual conduct 
of the training activities, we reviewed the written "Program of 
Instruction", the "Daily Schedule of Summer Training Program: 

July 1-August 9", and the "Preliminary Version of the Basic Program 
Plan-Program I: Specialist In Continuing Education"; listened to 
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selected tape transcriptions; and informally interviewed the 
training staff concerning program activities and the trainees' 
behavior o 



Although there were sufficient valid data to warrant analysis, 
certain constraints precluded a comprehensive assessment of 
the programc In section II, "Evaluation Constraints", we cite 
and discuss these constricting elements. 
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I. ANALYSIS: Instrumentation and Derivative Data 







I 



For contextual clarification, some general statements about 
the material presented in this section should be made. 

Eight of the 11 instruments were developed by the CERLI staff 
before the staff member making this report joined CERLI. Only 
one instrument- the Attitude Survey- had been previously tested 
on a comparative sample basis. 

The staff, some participants and this assessor considered the 
IITRI Communications Analyses Test, the Knowledge Test, the 
Attitude Scale and the Staff and Participant Evaluation Forms 
so lengthy and time-consuming that a testee’s motivation might 
change as he worked with the instrument. 

Although the ideas and techniques incorporated in the new 
instruments reflect a sophisticated approach to the evaluation 
of the training program, these instruments do contain some 
flaws that an opportunity to pre-test might have identified 
and corrected. 

In measuring participants’ behavior, the problem of allocating 
time for using instruments was critical. The Daily Evaluation 
Form somewhat served this purpose. However, on only two 
occasions (the first and second to last days of the program) 
were five of the instruments used to sample behavior. Only 
once (on the second to the last day) were three other instruments 
used to assess individuals’ reactions. 

Whether we can assume that data derived from instruments used 
in the final days of the program validly reflects participants’ 
reactions during the entire six weeks may be somewhat tenuous. 

In the following text specifically dealing with each 
instrument, the analysis will include an appraisal of the 
instrument’s usefulness. For example, such instruments as the 
Knowledge Test, Attitude Survey, Written Performance, Performance 
VTR and the IITRI Sociogram (administered on a pre and post 
basis) were designed to assess change in behavior over time 
and thus, by implication, would indicate behavioral change 
effected by the training program. To some extent, the Daily 
Evaluation Report measured attitudinal change effected by 
specific program activities. 
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The Trainee Comparison Form and the post test IITRI Sociogram, 
which do not measure change, reflect only the perceived 
relative position of the participants. The Staff and Participant 
Evaluation Exercises provide data about the adequacy of and 
satisfaction with the program^ 



1. Knowledge Test 

This instrument (entitled Background Information) sampled 
the participants’ knowledge of the program’s substantive 
content. Consisting of Forms A and B (each including 
31 multiple choice and 10 matching items) the instrument 
dealt with goal establishment , data collection , group 
processes , confrontation and search area . 

During the first week, half the trainees were tested with 
Form A and during the last week with Form B. The other 
group of participants initially was tested with Form B 
and finally with Form A. This scheme yielded comparative 
scores that would indicate how much content actually had 
been "'learned" or at least remembered. Such. content 
assessment could have provided valuable data for evaluating 
the training program o 

However, when we found that the two forms were non-equivalent 
or non-parallel we did not statistically analyze the 
results to determine change. This discrepancy probably 
occurred because there had been insufficient time to 
empirically test form equivalence before the training 
program was conducted and thus the forms were assumed to 
be parallel. 

To test this assumption post factum, 21 Wheaton College 
student teachers (a group more similar to the Charleston 
training group than a random sample drawn from the population) 
took a composite of both forms (even numbered items from 
Form A and odd numbered items from Form B) . Subsequently, 
this composite test was split and each student teacher’s 
scores on Form A and Form B compared. 

A Pearson Product Moment Correlation (Smith, 1962) calculated 
on the Form A and Form B raw scores yielded an jc = -.002, 
indicating that there was no relation between the two forms. 

In addition, a Spearman Rank Order Correlation (Smith, 1962) 
calculated on the ranks of student teachers for each form 
produced a r^ = +,09, indicating a low positive relationship 
between the two tests. 
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On the basis of these analyses, the two forms of the 
Knowledge Test were empirically determined to be not 
parallel and thus not amenable to pre-post comparison. 
In fact, the correlation coefficients infer that the 
two tests could not be much less equivalent than they 
areo 

Two other weaknesses of the Knowledge Test also render 
the value of the results suspect. 

One ; 14 of the 41 items on each form (34% of the test) 
dealt with data collection techniques and 10 of the 
41 items (24% of the test) dealt with the search area. 
Thus, 58% of the test dealt with only two of the POI's 
(Program of Instruction) five components and thus 
disproportionately represents the actual substance of 
the training program. 

Two; 29% (n = 12) of the Form A Items and 34% (n * 14) 
of the Form B items reflect material that was not 
presented during the program. 



Conclusion ; 

The Knowledge Test's weaknesses negate use of Its data 
to properly assess the program. However, a similar type 
of instrument, which would measure the incorporation of 
substantive content, could be developed for future programs 
in order to indirectly assess the kind and quantity of information 
that participants had learned. 
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Moreover, such a test should more proportionately deal with the 
Individual content areas in the program and the questions 
should be written by individuals highly skilled in test 
construction. 



I 



Finally, to eliminate the problem of non-parallel tests 
only one form should be used; and for the SCE program 
as now designed, a single form would suffice. 
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2. Attitude Survey 



This instrument assessed participants' attitudes toward 
educational phenomena and change processes. It Involved 
trainees in rating 20 words (such as "innovation", "education", 
"testing") as positive, negative or neutral. In addition, 
trainees rated 130 items on a six-step scale calibrated from 
"strongly agree" to "strongly disagree". 
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A previous evaluator had adapted this survey to a scale that 
another staff member currently is developing for CERLIo How- 
ever, at the time neither individual had clearly and empirically 
established the factors being measured and the subsequent 
means for scoring the scale o Consequently, the data obtained 
on the Attitude Survey administered at Charleston are not 
readily amenable to analysis o 



Conclusion g 

Although the data from the Attitudinal Scale administered at 
Charleston were not used in this assessment, a valid and reliable 
instrument can be developed for use in the training program » 

Such an instrument has been described on page A“=13 in the 
CERLI preliminary version of the Basic Program Plan; Program I: 
Specialists in Continuing Education o 

Whether the current and twice revised version of the scale is 
valid and reliable remains to be seeno 



3 0 Performance Video Tape Reoarded (VTR) 

Administered on a pre and post test basis, the performance VTR 
was designed to measure, in a role-playing situation, the 
participants® ability to operationalize goals <, use resources , 
snd communicate with other teachers o Trainees assumed the role 
of the SCE helping a teacher (staff interviewer) to solve a 
hypothetical classroom problem o 

On the pre test, half the trainees role-played in the presence 
of the CERLI staff member, while a second staff member 
simultaneously played the same role with the other trainees o 
By consistently asking appropriate questions and allowing time 
for trainees to adequately respond, both staff members played 
their roles so sophisticatedly that each trainee could be 
objectively scored on goal establishment, use of resources, 
data collection, and ability to communicate the problem to 
other teachers o 

On the post test, the same hypothetical problem was presented. 
Because the two staff members who had performed as pre tist 
interviewers also had actively participated in the entire 
training program and thus come to know individual trainees 
veil, the previous evaluator decided that in order to eliminate 
potential bias different interviewers should assume the teacher 
roleo 
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However, these post test interviewers (two university students) 
frequently failed to consistently ask appropriate questions and 
frequently interrupted trainees before they had completed their 
responses. The interviewers* inept techniques flawed the post 
test recordings and invalidated any objective or subjective 
scoring. Consequently, pre and post test data could not be 
meaningfully compared o 



Conclusion i 

Although we could not use the data from the Performance VTR 
administered at Charleston, we believe that this measure 
could become an effective device for assessment. In the next 
pilot test, we would suggest that one interviewer who is not 
actively involved with other phases of the program role play 
the teacher in both the pre and post testing sessions. 

Problem Awroaah 

This instrument Involved the participant in an open-ended written 
exercise posing a hypothetical classroom problem. In dealing 
with the problem, the trainee had to state goals operationally, 
utilize resources, and discuss ways to measure the attainment 
of goals. The Instrument, consisting of four items, was 
administered on a pre and post basis. 

In item one, the trainee stated and ranked five goals. (In our 
assessment, we did not score or use the rankings because the 
rationale for ranking was unclear.) We rated each goal statement 
on a scale of "operatlonallty": 0 » failure to operationalize 

the goal; 1 ■ unclear or ambiguous operational definition of 
the goal; 2 » acceptable statement. 

Statements from Preparing Instructional Objectives (Mager, 1962) 
were used as criteria for scoring this item. For each trainee, 
the goals were separately rated, summed, and a matched group 
t-test (McGulgan, 1968) performed on the sums between the pre 
and post tests. The post test mean of 5.71 was higher but 
not significantly higher than the pre test mean of 4.36 
(t * 1.35, df * 13, p < .20, 1 tail) which suggests that 
participants did not appreciably Improve their ability to state 
goals during the training program as measured by this 
Instrument and set of criteria. 

In item two, the participant selected five among 15 resources 
(names of organizations or titles of books) that he would 
consult in solving the given problem. Among the 15 items 
five were predetermined to be relevant; five. Irrelevant; 
and five, fictitious. 
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Regarding the relevant resources, the differences between the 
pre test mean of 2.79 (mean // relevant resources used) and the 
post test mean of 3.21 were not found significant (t » 1.55, 
df « 13, p < .20, 1 tail). These minimal differences indicate 
only a slight tendency to use more relevant resources at the 
end than at the beginning of the training program. 
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For use of Irrelevant sources, the post test mean of .60 was 
(as would be expected) not significantly less than the pre test 
mean of 1.20 (t ■ 1.18, df » 4, p < .15, 1 tail). Though this 
finding Implies that participants selected fewer irrelevant 
resources at the end of the training program, one should note 
that on both pre and post tests the means were very low and 
that had a post test mean of 0.0 been attained the difference 
between the two means of 1.0 still would not have been 
statistically significant. 

For selection of five fictitious resources, a mean of 1.83 
was attained on both pre and post tests. This result also 
suggests that the training program did not affect the partici- 
pants' choice of fictitious resources as measured by this test. 

In item three, the participant explained how he would know when 
each of his five goals had been reached. We rated each 
explication for its "objectivity" on the zero to two scale used 
in rating the "operatlonality" of a goal statement. (A statement's 
"objectivity" was judged by its substantive presentation of an 
objective measure of the goal stated in item one.) 

For each trainee, the statements were summed and the pre and 
post sums compared. A matched group t-test showed that the 
difference of 2.50 between the post test mean of 6.50 and the 
pre test mean of 4.00 was significant beyond .01 (+ « 2.96, 
df 13, 1 tall), which suggests that as measured by this 
test participants had Improved their ability to measure goals 
at the end of the training program. 

In item four, participants were to make additional comments. 

On the pre test, eleven individuals made such comments but 
on the post test only three dealt with this item. One could 
attribute this fact to the participants' fervid desire to 
complete the battery of tests. 



Conclusion: 



Data from the Problem Approach Test seems to suggest that 
although the training program did not significantly affect the 
participants' ability to define goals and utilize relevant 
resources, It did positively Influence their capacity to 
measure goals. 
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However, these statistical analyses may not necessarily or 
accurately reflect only the program’s effects «. Like other 
instruments, the test itself is a variable that may influence 
the interpretation of data» For example, a participant might 
have mere precisely stated his goals in item one if he had 
been instructed to implement independent study "in your own 
area" and thus formulate his response more specifically in 
terms of his own academic interests <> 

In item two (resources), all the organizations' names should 
have been completely listed rather than some designated by 
acronyms. Furthermore, space for a participant to indicate 
his own relevant resources should have been provided. 

Despite these weaknesses, the data of the Problem Approach Test 
could be regarded as a valid measure. In future training 
programs, this type of exercise— if appropriately revised — 
seems a useful and practical means for determining whether the 
participant has in fact acquired the skills of goal establish- 
ment, data collection, etc. 



5. Trainee Comvarieon Form 



This Instrument provided a peer ranking technique for staff and 
participants to individually compare participants according to 
their perceived future success as an SCE, By pairing each 
participant's first name once with every other participant's 
first name, a listing of 91 different pairs was made. From this 
roster, staff and participants chose that member of the pair 
that they judged "most likely to succeed as an Inservlce leader 
according to your understanding of the inservice leader role". 

After the preferred names had been tabulated, they were scaled 
in accordance with the method of Pair Comparisons (Guilford, 1954; 
Edwards, 1937): on an interval scale, equal differences in the 

numbers assigned to the participants represent equal differences 
among participants in the perceived likelihood of success. In 
order to bring the lowest ranked participant up to a value of 
zero, a constant was added to each scale value. For this 
reason, the Integral differences between numbers rather than 
the actual numbers becomes significant. Because staff and 
participants separately completed the Trainee Comparison Form, 
two separate scales were derived. A scale combining staff and 
participants was not calculated because the greater number 
of participants would have differentially weighted the scale 
values. 
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Certain features of the scale values graphed on Figure 1 (p, 9) 
should be noted. The participants' ranking produced scale 
values that rated one participant exceptionally high, three 
participants relatively low, and ten participants clustered 
in the middle range. The staff's ranking produced scale 
values that rated participants more equitably on the entire 
continuum. A Pearson product-moment correlation between 
the staff s and the participants' scale values yielded an 
r ■ + ,82 (p < .01), which indicates high positive agreement. 



Conclusion; 



Because the Trainee Comparison Form was designed to indicate 
relative positions, its data do not permit inferences about 
a participant's knowledge or mastery of the program's content. 
Even if this form had been administered on the first or 
second day of the training program, the participants' relative 
positions might have been similar if not identical. 

Although this measure yields more quantifiable data than any 
other Instrument used in the program, it Is primarily valuable 
as a predictive indicator of success in the field. For this 
reason. It would seem wise to continue its use in future 
programs. 
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6. IITRI COMMUNICATIONS ANALYSES TEST 



The Illinois Institute of Technology Research Institute (IITRI) 
was contracted to develop a conununlcatlons analysis test that 
would accomplish the following objectives? 



The theoretical framework for this Instrument was based on research 
conducted by Nan Lin ( a sociologist from Michigan State University) 
concerning the application of soclometrlc techniques to the study 
of educational Innovations In secondary schools. Though IITRI 
did not design the Instrument specifically for the CERLI 
training program, the previous evaluator thought that It could 
be pilot tested at Charleston. However, In the context of the 

following IITRI statement, the validity of the Charleston data 
Is questionable: 











Analyze the communications and attltudlnal Impact 
of the summer training program sessions for 
specialists In continuing education (SCE) con- 
ducted by CERLI on the trainee group. 



Analyze the communications and attltudlnal Impact of 
the SCE's seminars upon the teachers In his school. 



Analyze the correlations between the attltudlnal 
variables and communication role attributes. 



% 



the Instrument used for the training group surveys 
was originally designed to be used In the school 
environment. Its questions seek to determine the 
group member's relations to his peer group and to the 
group's superiors. Many of these questions lose 
meaning when taken out of the school context and the 
validity of the Instrument as a measure of group relations 
thereby suffers. 



I 






Heterogeneity of the training group Introduced the 
problem of non-parallel response to the questions. 

A superintendent— of —schools would view his relations 
to a principal (or even the more general category of 
superior) much differently than an elementary school 
teacher. In addition, heterogeneity and unfamlllarlty 
of the members of the training group led to meaningless 
soclometrlc responses on the pre-event survey. The 
soclometrlc data on the post-event survey, while 
showing which Individuals were opinion leaders (useful 
for correlation purposes) did little to show change In 
communication patterns caused by the training sessions 
(because of the lack of valid pre-event Information.)'" 
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Even though mls-appllcatlon of the IITRI instrument virtually 
nullifies the data's interpretation, an analysis of responses 
to question 70 resulted in the useful post test sociogram shown 
in Figure 2 (pJ.3). By responding to "Among the teachers 
participating in this training seminar, name three whom you 
respect as teachers", the 14 participants revealed their per- 
ceptions of the communications network at the end of the 
training program. Although four of the 14 participants emerged 
as opinion leaders, all but one (//4) had been identified as 
such on the pre test sociogram at the beginning of the program. 

(In the previously quoted IITRI statement, the questionable 
validity of the pre test sociogram is noted.) 

Correlations of staff and participants' choices indicated in the 
post test sociogram with the scaled peer ratings predictions 
shown in Figure 1 were calculated. On the sociograms, rankings 
were based on the number of times a participant was chosen by 
another participant. Spearman rank-order correlations yielding 
an r<# ^ .82 (p < .01) between participant's sociometric and peer 
rating measures and an r * +.57 (p < .05) between participant's 
soclometrlc and staff peer ratings suggest that a participant's 
perceived opinion leadership in the training program is highly 
positively related to his predicted success as an SCE. Since 
opinion leadership (as defined by question 70) could be considered 
a successful SCE's desirable attribute, these correlations were 
not unexpected. These correlations also provide a basic form of 
convergent validation of peer ratings and the post test soclo- 
metrlc analysis techniques. 

The post test sociogram shows two well defined cliques or sub- 
groups whose members maintained extensive communications with 
participant //13, the group's perceived leader. There also is 
some evidence of crossing communications between participants 
//12 and //16 and between //2 and //4. In our assessment, we attempted 
to find out if members of these respective cliques differed in 
their participation in and response to the training program. 

We compared members of the two sub-groups with the staff and 
peer rankings shown in Figure 8. A Wa Id-Wolf owl tz ^uns test 
(Siegel, 1956) — calculated to determine if the cliques tended to 
fall on either end of the success scale — yielded nonsignificant 
results for both the staff and the participants' rankings. Thus, 
as defined by peer ranking scales, differences in the cliques' 
membership were random. 














We then discussed the subgroupings with the training staff who 
interpreted the differences as aspects of social behavior. One 
clique (participants i/3,//4,//6,//ll,//15 and //16) included some 
individuals who commuted from their homes each day and some 
individuals who seldom participated in the group's social 
activities. According to the staff, members of the other clique 
(participants //2,//5 ,//7,//9,//10,//12 and //14) attended most of the 
social functions held during the training program. 



Conclusion ; 

Due to validity problems, the IITRI communications analyses 
test provided only a limited amount of information. However, a 
post test sociogram did show participants' communications patterns 
that reflected a hl^h positive correlation with the Trainee 
Comparison Form's scale values. 
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7. staff Evaluation Exercise 

Appendix B Includes the Staff Evaluation Exercise and the means, 
standard deviations and tabulations of data for five staff members 
who completed the form. Ratings were made on such variables as 
satisfaction, success and adequacy. Unfortunately, no statements 
of significance levels could be presented because criteria for 
the relevance of such variables to the program had not been 
predetermined. 

The following statements summarize the most relevant data: 



1) The professional staff was unanimously satisfied 
with preliminary Information provided about most 
aspects of the program although the staff would 
have found some additional Information helpful. 
(Questions 1~3.) 

2) The staff believed that the program's objectives 

(as defined by the staff) had been almost completely 
achieved. (Questions 4-8.) 

3) The staff expressed satisfaction with such activities 
as practice teaching, practice leader seminars, 
staff critiques of practice leader seminars, lecture 
and general sessions, search area, evening groups 
and social activities. Less satisfactory were the 
housing accommodations. (Questions 10-24.) 

4) Concerning the amount of time allocated to the same 
activities In future programs, the staff. In most 
Instances, suggested that: 



a. Less time be spent on practice teaching and 
general lecture sessions 

b. More time be devoted to the search area, social 
mixing, leader practice and leader seminars 

c. More time be released for "free time" 

d. Approximately the same time be spent on staff 
critique, seminars and evening groups (Question 26.) 



5) 



The staff generally classified coverage of the program's 
substantive content as "about right" but somewhat 
"general" rather than "detailed". Coverage also was 
rated about half way between "Insufficient" and 
"about right". On the whole, the staff considered 
substantive coverage satisfactory. (Questions 27-28.) 
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6 ) 









The staff felt that the exchange of information and 
concepts among "fellow members" of the program approached 
the maximum (Question 29) and that in meeting group and 
individual needs the balance achieved by the program 
approached excellence. (Question 30.) 



7) With the exception of social fraternization with the 
group, the staff felt that the program's many aspects 
had been adequately or more than adequately administered. 
(Questions 32-45.) 



8) The staff's perceptions of the most and the least 

effective parts of the program, activities that should 
be eliminated or added, follow-up activities and 
summaries of the staff's personal reactions are 
presented in Questions 46-55, Appendix B. In general, 
the staff seemed very satisfied with the program as 
a whole. 



Conclusion ; 

Responses to the Staff Evaluation Exercise probably should be 
regarded as relatively accurate and valid assessments of the 
Charleston Training Program. As an Integral part of that program, 
each member could evaluate the program as a series of components 
and as a whole. The summary statements consistently affirm the 
staff's favorable impressions of the program. 

Admittedly, the staff's close Involvement with the program might 
have Influenced their views. (Rosen thall (1966) presented a wealth 
of evidence demonstrating the pervasiveness of such bias.) How- 
ever, it would be difficult to assess to what degree bias affected 
the staff's judgments. During the interviews, they explicitly 
expressed an awareness of the bias problem. Such awareness 
probably is the most effective means of eliminating or at least 
controlling such bias. (Kelman, 1967.) 



8 . Partiaipant Evaluation Exercise 

Appendix C Includes the Participant Evaluation Exercise and de- 
tailed data and responses. The following statements summarize 
this material. 

1) Participants generally felt that the program 
achieved its objectives as they defined them. 
(Questions 9-12, 20.) 
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2) Participants generally expressed confidence In their 
ability to conununicate Ideas, adapt to a new situation, 
work well with others, make decisions, understand and 
be sympathetic toward others, and subordinate self 
Interest for the sake of group Interest. 

3) Participants generally expressed satisfaction with the 
practice leader seminars, staff critiques of these 
seminars, evening group sessions, the search area. 

Informal social activities, housing and meals. They 
expressed less satisfaction with practice teaching 

and the lecture and general sessions. (Questions 24~40.) 

4) Concerning the amount of time allocated to the same 
activities In future training programs, participants. 

In most Instances, suggested that: 

a. Somewhat less time be scheduled for lecture sessions 

b. More time be scheduled for leader practice, seminars, 
and free time 

c. i!|>proxlmately the same time be spent In social 
mixing, evening groups, search area and practice 
teaching (In a previous response they had seemed 
less satisfied with this activity) (Question 41.) 

5) Participants generally felt that coverage of the programme 
substantive content (goal establishment methodology, 

data collection, etc.) was "about right" In terms of 
excessive or too Insufficient, too general or too detailed, 
too simple or too advanced. (Question 42«) 

6) Participants generally felt that the exchange of in- 
formation and concepts was slightly more than half way 
between mlnlmimi and maximum. (Though this rating of 
the training program may seem somewhat mediocre. It 
should not be considered unfavorable.) (Question 44.) 
Participants generally felt that In meeting group and 
Individual needs, the balance achieved by the program 
might be considered good. (Question 45.) 

7) Participants generally rated the staff's administration 
of the program (arranging and executing the schedule, 
maintaining an objective relationship with participants, 
helping group members with various needs, resolving 
conflict among group members, providing resources and 
facilities) between "extremely adequate" and "adequate". 
(Questions 48-61.) 
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8) Participants generally felt that most of what they had 
learned in the program would help them to function 

as SCE’s in their schools. (Question 62.) 

9) The participants' views about the program's most and 
least interesting aspects and the activities that 
should be eliminated or added are presented In 
questions 63-66, Appendix C. 

10) Participants' additional Impressions of the program, 
desired follow-up activities, personal reactlc^as and 
predictions about potential problems In their schools 
are presented in questions 67-75, Appendix C. 

In general, participants seemed to be satisfied with 
the program. 



Conclusion: 






There Is no reason to believe that these data do not validly 
indicate participants' perceptions and reactions. 

These findings, which seem to reveal the training program's 
strengths and weaknesses, should provide a valuable source for 
assessing the effects of the CERLI Training Program and for 
planning, refining and administering the next pilot test. 



9. '‘ Partioivant Re-Evaluation Exercise 

Appendix D Includes this Participant Re-Evaluation Exercise and 
relevant tabulations and statistics. Approximately five weeks 
after the completion of the training program, participants were 
sent this exercise which ten completed and returned. (Participants 
//4,//5,//ll and //16 did not respond.) 

Not Included in the original Charleston evaluation battery, this 
Instrument (consisting of 37 Items selected from the Participant 
Evaluation Exercise) was designed to check the reliability of the 
participant's previous responses and to determine the trainee's 
general attitude toward the training program at a time when he was 
preparing to actually function as an SCE. Substantially the same 
responses to questions a participant had previously given were 
regarded as "reliable" reactions. 

The following statements summarize these reactions: 
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1) Although participants continued to express confidence, 
their level of confidence showed a decrement — 
especially In the ability to adapt to new situations , 
to work well with others, to view problems In a broad 
sense, to subordinate self Interest for the sake of 
group Interest and to make friends. (Question 1.) 

This decrement may represent an actual change In self 
perception. However, the fact that many were waiting 
for administrative decisions about their opportunity to 
function as an SCE may have tempered their "revaluation" 
response to this question. 



I 2) The participants' re-assessment of allocated time for 

i each substantive component In the next training program 

i for the most part duplicated their original response. 

I (Question 9.) 

^ 3) The participants' re-assessment qf coverage of the pro- 

^ gram's substantive content resembled their previous 

response except that they now believed that confrontation 
techniques and group functioning had been "too general" 
and that goal establishment, data collection, resource 
methodology, small group processes, confrontation and 
I group functioning had been "insufficient". (Questidiv 10.) 

4) The participants' re-assessment of the administration 

of the training program was essentially similar except 
that they now generally believed that staff social 
mixing and resolution of conflicts among group members 
had been more adequate than they originally had perceived 
them to be. (Question 15-27.) 

5) Concerning anticipated proibltt&s In implementing the 
Inservlce seminar, participants' predictions were 
essentially similar except that they now identified 
more difficulties created by pressures of regular 
duties, colleagues' resistance to change, and their 
own feeling that the training had been inadequate. 
(Question 34.) 

6) In Identifying three individuals who they thought would 
be most successful SCE's, participants generally agreed 
with the scale values shown in Figure 1 except that 
//14 now ranks second rather than fifth on the scale. 

(See question 38, Appendix D for the listing and 
tabulation of nominations.) 
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7) In identifying three individuals who they thought 
would be least successful SCE's, participants 
generally agreed with the scale values shown in 
Figure 1« (See question 38, Appendix D for these 
results. ) 



Conclusion : 

In assessing the perceived success of the program, we compared the 
original staff and participants’ mean responses to the 13 questions 
in the Re-Evaluation Exercise that required rating on a continuum. 

(See Appendix E.) 

On items 1-12 (the staff did not answer //13) , the staff perceived 
the program more favorably. Since the staff planned, developed, 
administered, and was totally involved in the training program, 
these favorable perceptions would seem predictable. However — 
since no criteria for perceptual accuracy can be established, there 
is no reason to believe either that the staff was unduly optimistic 
or the participants unduly pessimistic about the program's perceived 
success . 

That each of the staff's and the participants' ratings fell within 
a range no greater than one fourth of the scale's spectrum probably 
reflects verdical perceptions and validates the accuracy of the 
ratings. It also should be noted that eight of the 12 re- 
evaluation mean responses fell between the staff's and the participants' 
ratings which suggests that — five weeks after the training's con- 
clusion — participants' perceptions tended to move toward the 
staff's perceptions. 

Conversely, participants' satisfaction with the search area and 
the small group sessions decreased slightly over time as did their 
perception of the balance achieved by the program in meeting group 
and Individual needs. The meaning of these last two findings is 
not clear. 



10. Daily Evaluation Report 



In this daily report (called "Brickbats and Bouquets") , participants 
indicated (1) the most meaningful activity, (2) the least meaningful 
activity, (3) the disliked activities or "brickbats", (4) the 
liked activities or "bouquets". Reports were submitted on Monday, 
Tuesday, Wednesday and Thursday afternoons and the responses 
categorized and duplicated for staff members each day. 
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For our assessment of the training program, we separately tabulated 
and summarized on a week-by-week basis the responses to items 1 
and 2 dealing with the major dally activities listed in the training 
schedule (general session, seminar, practice teaching, search area, 
T-groups). Originally social events was classified as a major 
activity, but so few responses fit this classification that the 
category was deleted. Because of the great number, diversity, and 
contextual nature of the participants' responses to items 3 and 
4, it seemed Impractical to quantify, summarize or Interpret these 



Figures 3-9 plot the participants' dally "evaluations" of pro- 
grammed activities. In Figure 3, (p.22), one should note that 
T-groups and practice teaching sessions do not "show" during the 
first week because these activities were not scheduled during the 
first week. Also, the staff conducted small group seminars only 
on the first two days of that initial week. 

The fact that for each of the five activities, the greatest number 
of most meaningful responses occurred during the program's 
first two weeks may seem curious. Whether a diminishing enthusiasm, 
increasing pressures or other factors account for these peaks 
and subsequent decline can only be speculated. One can state, 
however, that the seminars generally remained the single "most 
meaningful activity In the training program. 

In figure 4, (p.23), no one activity consistently was cited as 
least meaningful". That, in general, all the activities were 
meaningful to the participants may be Implied. 

In Figures 5-9, the "meaningfulness" of each activity is plotted. 
Although most meaningful" outnumbers "least meaningful" responses. 
Figures 5, (p.24) and 6 (p.25), indicate that the general sessions 
and the seminars were most often cited. Conversely, the T-groups 
(Figure 7, p.26) elicited a less than moderate number of "most" 
and least meaningful" responses. Figures 8 (p.27) and 9 (p.28) 
show that the search area and practice teaching generated 
fewer "most" or "least" responses. 



Why these respective differences in responses occurred cannot be 
readily determined. Why both the general and the practice teaching 
sessions seemed to have been "less meaningful" as the training 
evolved also poses a ponderable. The fact that participants 
perceived the search area as neither a "most" or a "least meaningful" 
activity could be attributed to their tendency not to regard this 
regularly scheduled two-hour period as a specific activity. 
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Conclusion; 



Probably the training staff could most validly interpret the 
data presented in Figures 3-9, which does provide potentially 
significant Information for planning and operating future 
programs. As a type of on-going assessment device, the daily 
evaluation form certainly has merit. 



- 21 - 




m 






eral Session 












|er|c 



..,^* a^f. r^'m*' ' » I * »i.« 



i>*^A,*vv4 y •4.yh*ir tvwr tv 




•5^33^1 J3<J S3SU0dS3^ JO lUng 









cu 






r-i 


c\3 


e 






ri 




•H 






iW 


CO 


H 


CO 




60 


ci3 




OJ 








U-4 


CO 




•H 


CO 


o 


c 




d 


(U 




o 


r) 


cd 


•H 




p- 




<U 




o 


co 


d) 




•H 


•H 


o 


u 




> 




Pi 


:3 


u 


•H 


U 


. 


W)' 


CO 


4J 


d 


‘TJ 


•H 


o 


O 




d 




?3 


<J 


rx^ 


d 



-: 22 - 






I 






I 



U 



1 



t><;5 



■J 



4 









) 



j 






General Session-H- ; - ! i i n is;; 









< >« Ml >4 .(MtfcllV *« • . <«,>« ki*4<VM4r<*.*4-'^««k ' 







CO 

K* 



o 

o 



c 

•H 



0 ) 

g 

•H 

H 



to 

G 

•H 



•H 

CO 

J 4 

H 





iH 




cu 






G 


G 


g 






CM 




•H 






to 


CO 


H 


CO 




G 


G 




0 




•H 




U-^ 


CO 




G 


CO 


o 


G 


<T 


03 

CU 


cu 

•H 


G‘ 


o 

G. 








O 


CO 


GJ 




•H 


•H 


0 




u 


> 


4:i 


g; 


G 


CO 


•H 


a 




tr 


G 


U 


G 


TJ 


•H 


C) 


O 


G 


G 


PmI 




< 




ttj 



- 23 - 




inii 






ii 






!] 



i) 






'^1 



il 


















S( 



l■■ 









K 









F- if«l(«M«Ui t> MMm aHMi Mt|^<(M^t>. d 



iH 

P 

^ 4-4 

04} 

P 

•H 

P 

P 

d) 



p 

00 

p 

•H 

P 

P 

a 



4J 

CO 

o 

S 



u 

CO 

p 

p 



-vO 



> 



-in 



/ 



/ 



/ 



/ 



/ 



/ 



-•^ *H 



/ 

V 









w 

o 

CJ 



p 

6 

•H 

H 



00 
P 

-cn *H 

p 

•H 

P 

’ H 



CNi 



— I : r 

CN O 00 

CO 



CO 



cs 



vO 

cs 



sf 

cs 



cs 

cs 



o 

cs 



CO 



VO 



I 

sr 



cs 



00 



VD 



CS 



J3<I sasuodsa^ jo uing 











(0 










p 








•Td 


CO 








p 


p 






CO 


p 


o 






P 




04 




O 




p 


CO 






p 


•e 


p 




CO 


o 


•H 


pc! 




CO 


•H 


H 






p 


CO 




4-1 




p 


CO 


4^ 


o 




rH 


P 


O 






P 


CO 






m 


^-1 




P 


o 




to 


iH 


O 


p 


CD 


p 


P 


•H 


p 


5^ 


•H 


u 


U 


p 


P 


P 


p 


O 


cr* 


W 


P 


p 


P 


P 


•H 


P 


p 


P 


u 




s 


o 







- 24 - 

















■ ,*„ . . • . ' » . < . ■ i" Y 







(in weeks) 







Training Time (in week 




FavtiaiTpant Change Suwen 

Foif additional data concerning change in participants’ behavior 
during the training program, we asked the staff to submit this 
Participant Change Survey* Although this instrument was devised 
and administered four months after the training program had 
concluded, we needed to collect such information about the staff’s 
perceptions which had not been collected during or immediately 
after the program had terminated. 

Even though post factum and admittedly weak and unsophisticated, 
the questionnaire did serve a useful purpose. For one thing, the 
training staff in their dally involvement in programmed and non- 
programmed activities became acquainted with most of the trainees 
and thus could offer valid reactions and perceptions about 
individual participants. Secondly, since some of the key instru- 
ments have been shown to have failed, no other source has yielded 
the kind of information this survey provided. The rationale for 
using this questionnaire outweighs the less acceptable feature — 
that the staff’s ego involvement with participants might distort 
inferences drawn from the Participant Change Survey. 

The staff rated each of the 14 participants’ "change in behavior" 
toward or away from their perceptions of the successful (not 
necessarily ideal) SCE. Each staff member completed two sets of 
ratings: "change" as perceived at the end of the program, that 

is over the six weeks; and "change" as perceived from the 
program’s termination to the time of completing the questionnaire. 

On each set of ratings ("August" and "now") the staff also 
indicated the amount of contact with each individual as "none", 
"little", "some" or "much". (Since most of the contact responses 
fell in the "some" and "much" categories, no special weightings 
were assigned to the ratings.) The ratings were made on a 
7-point scale ranging from a -3 ("great" negative change) to a 
+3 ("great" positive change) with 0 representing "no change" 
and the 1 and 2 ratings denoting "slight" and "moderate" change, 
respectively. The questionnaire does not tap the types of changes 
perceived nor operationally define positive and negative change. 

In Table I, (p.31), the mean ratings indicate that 12 or the 14 
participants (86%) showed positive change; one, no change; and 
one, slight negative change during the six weeks' training period o 
Six of the 12 (50%), who changed, moderately or more than 
moderately changed in the direction of the successful SCE. 

Mean ratings of perceived change from August to December again 
indicate that 12 of the 14 participants changed positively with 
el^ht of these means falling between "slight" and "moderate". 

The two individuals perceived as showing no change were the 
same individuals who showed zero and negative change in the 
August rating. Due to lack of contact between some staff and some 















SCE's, the number of raters in the second set of ratings differs 
from the original number. 



Conclusion : 

The participant Change Survey suggests that the staff perceived 
most participants as having changed positively in the direction of 
the successful SCE both during the six weeks' program and at 
the time (December) they again rated them. 

Although such variables as history, maturation, restricted value 
of the ratings, and the time lapse between the two sets of ratings 
do qualify any interpretation of the survey's results, this data 
could be regarded as useful because it does reflect the staff's 
perception of participants' behavior. 







5W.^;^FV4t'A?S55.-.^ 












TABLE 1 

MEAN RAT'^NGS OF PARTICIPANT CHANGE 
BY STAFF 

DURING AND AFTER TRAINING 



i 




Participant 

Number 


Number 

of 

Raters 


Mean Rating 


Participant 

Number 


Number 

of 

Raters 


Mean Rating 


2 


5 


+1.80 


2 


2 


+ .50 


3 


5 


+2.00 


3 


3 


+2.00 


4 


4 


+2.00 


4 


2 


+1.00 


! 5 

t 


5 


+2.00 


5 


4 


+ .25 


6 


5 


+ .80 


6 


3 


+1.67 


7 


5 


+2.20 


7 


3 


+ .67 


9 


5 


+ .80 


9 


4 


+1.00 


10 


5 


+1.40 


10 


4 


+ .75 


11 


5 


+ .20 


11 


2 


+1.00 


12 


5 


+2.20 


12 


3 


+1.67 


13 


5 


+2.20 


13 


4 


+1.50 




4 


+ .75 


14 


3 


+1.67 


15 


4 


0.00 


15 


2 


0.00 


16 


5 


- .75 


16 


3 


0.00 
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Summary 






Analyses of data derived from the 11 Instruments have shown: 

A. Data derived from the Knowledge Test, Attitude Survey, 
and VTR could not be used because the Instruments 
were unreliable. 

B. Data derived from the Problem Approach Test (which 
exhibited minor flaws) showed change In participants' 
ability to measure goals but did not show change In 
participants' ability to establish goals or to use relevant 
resources. 

C. Data from the Trainee Comparison Form showed high positive 
correlation between the staff's and the participants' 
ratings of the participants' perceived success. 

D. Data from the Staff Evaluation Exercise and the Participants' 
Evaluation Exercise showed that both groups held similar 
views about their satisfaction with the activities, content 
and conduct of the training program. 

E. Data from the Participants' Re-Evaluatlon Exercise showed 

a consistent similarity to the participants' first assessment 
and a general agreement (reflected In both evaluations) 
with the staff about certain key Items. 



F. Data from the Daily Evaluation Report show|d that no 
activity was generally regarded as "meaningless" and 
that the small group seminars were generally regarded 
as the "most meaningful" activity. 

G. Although the IITRI Communications Analysis Test had been 
inappropriately used in the training program, the 
sociogram showing the communication structure correlated 
highly with the Trainee Comparison Form's peer ratings. 

H. Analysis of the Participant Change Survey indicated that 
12 of the 14 trainees changed favorably and that the 
change in eight participants was rated "moderate" or 
greater. 
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II. 



1 . 



EVALUATION CONSTRAINTS 



As stated In the Introduction to this Assessment of the CERLI 
Training Program for SCE's, certain constraints precluded a 
comprehensive assessment of the program even though there 
were sufficient data to warrant the analysis presented In 
section I of this report. 



In this section, we present a brief explication and comments 
about those constraints and suggestions for eliminating or 
counteracting or alleviating their effect on future training 
programs . 



Lack of sufficient loncf-rancie^ follcw-uv data to assess 
the SCE*s effectiveness 



Whereas the training program Imparts the skills needed by an 
SCE, the degree to which he has acquired these skills may not 
be adequately measured until he applies them In his work with 
and Impact on those Individuals participating In the SCE's 
seminars . 



Currently, CERLI Is programming follow-up activities Including 
the study of the SCE's Impact. By pre and post testing teachers' 
attitudes and communications, CERLI Is conducting this Investigation 
In schools where SCE's are functioning. In addition, both the 
SCE's and their seminar participants submit questionnaires after 
the seminar sessions and data derived from this Instrument are 
being processed. This follow-up phase Is scheduled for 
completion In the Spring, 1969. 



In order to find out If the CERLI program effectively trains 
SCE's, Information about their actual functioning as an SCE seems 
requisite. This data also provides feedback to the staff and 
thus enables them to appropriately modify the training alternatives. 



2. Failure of some of the key instruments desicpted for 
measupim oharuje to yield valid data 



If the Knowledge Test had yielded valid data, whether or not 
participants had remembered segments of the substantive content 
could have been statistically determined. 



If the pre and post administered Attitude Survey had yielded 
valid scores, changes In participants' attitude toward educational 
phenomena could have been detected. 
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If the pre and post administered Performance VTR (Video tape 
recording) had yielded valid data, any change in the participants' 
ability to operationalize goals and to use resources could 
have been identified. 

In CERLI's SCE training program, each of these instruments 
constitutes a useful assessment device and could, if properly 
designed and written, be used In future programs. 

The assessor's lack of involvement in the trainincf y^ogram 
and post faetum appointment to the CERLI staff 

Because the assessor came to CERLI two weeks after the 
Charleston program had terminated, he has had little or no con- 
tact with individuals whose degree of behavioral change is a 
subject of this report. Among the 14 participants, he personally 
met, on one occasion, eight; he talked briefly with four 
participants on the telephone; with two, he has had no contact. 
Although program activities were measured by written Instruments, 
each (except the Dally Evaluation Report) sampled behavior on only 
one or two occasions. 

The training staff has provided valuable descriptions of the 
training program and insights about the participants' behavior. 
Nevertheless, without sufficient personal and first hand contact 
during the training period, any claim to having properly 
evaluated change in participants' behavior would be suspect. In 
order to understand the program's activities and to observe 
participants' behavior, it would seem logical to assert that the 
evaluator should experience the program first hand. 

In lieu of the assessor's personal experience, some systematic 
records of significant T-group experiences, programmed and un- 
programmed social activities and other relevant nonprogrammed 
events and incidents that occurred would have provided 
supplementary but meaningful information and perceptions. 

Although the Dally Schedule, written Program of Instruction, 

Daily Evaluation Report and information derived from the staff 
generally and sufficiently document the Charleston Training 
Program, the staff has suggested that Informal, unprogrammed 
activities Influenced not only the participants' behavior but 
the staff's perceptions of the participants. 

To provide this kind of supplementary data, it would seem 
feasible to designate at least one observer who would keep a 
diary of relevant programmed and nonprogrammed events and 
activities during the next pilot test. 
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4. Lack of vveoiselii specified ovitevia to evaluate a 
successful SCE's behavior 



Regarding evaluation criteria , Stake (1967) points out, 

"The evaluator needs a battery of standard operating procedures. 
Procedures depend on criteria; criteria depends on rationales; 
rationales depend on theories." 

In his "CERLI Assessment Report" (1968) , Sorenson notes that 
the development of criteria measures Is a necessary operation 
Involved In the analysis of program outcomes. 

Because the Charleston Training Program was only the second 
pilot test of a research and development project, the procedures, 
rationale and criteria were (as would be expected) only In an 
evolutionary stage at that time. Some weeks later, general 
statements of the criteria for a successful SCE's behavior were 
presented In the Preliminary Version of the Basic Program Plan- 
Program I: Specialist In Continuing Education. This disparity 

In timing prevented the staff from Incorporating Into the 
evaluation plan of the training program the criteria for 
evaluating participants' performance at the end of the training 
program and In the field. The procedures, rationale and criteria 
presented In the preliminary version of the Basic Program Plan 
now provide a substantially stronger foundation for establishing 
evaluation criteria for subsequent pilot tests. 

However, one should not Infer that there are no criteria for 
determining the success or failure of the SCE. Earlier documents 
of the program contain general criteria. Furthermore, the pro- 
fessional staff's experiences have created a pool of Information 
regarding the SCE's success. Nevertheless — precise specifications 
of the criteria for the evaluation, because of insufficient 
follow-up data and the necessarily continuous updating of this 
research and development project, remain Incomplete. 



5. Lack of logical methodolocu to establish cause and effect velations 

In discussing this "constraint", we will first deal with the 
evaluation of the effects of the various components (T-groups, 
data collection, goal establishment, etc.) and secondly with the 
evaluation of the training program's general design. 



As administered in Charleston, the training program may be viewed 
as a composite of numerous and various activities within which no 
formal attempt to ascertain their respective effects upon 
participants' behavior has been undertaken. In order to provide 
answers to such questions as "How much 'ilnpact does component X 
have on the participant?", "Is component Y necessary?", "What 
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effects do components X and Y have on component Z?", the effects 
of each component should be separately assessed. (Responses to 
the staff and participant evaluation exercises do provide In- 
direct answers to all these questions, but cause and effect 
relationships cannot be established from the data offered In 
this training program.) 

Each training activity or component could be Independently 
evaluated by sequencing coverage and concentrating on only one 
activity or component at a time. Not only would such a formal 
component evaluation permit approximations to establishing cause 
and effect relationships but It also would enhance efficiency 
and economy In designing and administering the program by 
Identifying activities that should be modified, dropped or added. 

That the SCE training program's design should fit all the rigid 
requirements of a laboratory experiment certainly is not 
mandatory, but In order to establish cause and effects certain 
elementary controls must be Incorporated. 

For example, the Charleston Training Program's design could be 
diagrammed as 0 X 0: one group of participants was pre- 

tested (0), received training (X), and v;as post-tested (0). 

Claiming that participants ' change In behavior was the result 
of the training program may be fallacious for, according to 
Campbell and Stanley (1963) , a number of alternative suggestions 
exists solely because of the design's nature. In this design, 
behavior change might be attributed to: 

(1) Historical and maturatlonal factors occurring 

during the six weeks 

(2) Effects of testing and use of Instruments 

(3) Selection of participants and reasons for mortality 

(Actually, only one partlcl||ant dropped out 
of the Charleston Training Program.) 

(4) The peculiar Interaction of any of these variables 

Controlling or eliminating these sources of Invalidity probably 
would Increase the likelihood of arriving at a valid Interpretation 
of the findings. For example, not only an experimental but a 
control group should be patterned In the SCE program. Members 
of the control or placebo group would be randomly drawn from 
the population of applicants selected for training. These 
control or placebo group Individuals also would be pre and 
post-tested on the Instruments administered to the experimental 
group . 
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After incorporating these controls, the SCE training program's 
design would be a Pre-Test Post-Test Control Group Design 
(Campbell and Stanley 1963) diagrammed as: 

R 0 X 0 
R 0 0 



Each row represents a treatment group: X denotes the group 

receiving SCE training; the omission of X indicates the 
I control group receiving either placebo or no training. 

I R symbolizes the random selection and assignment of individuals 

I to a group. 0 symbolizes the testing. According to Campbell 

I and Stanley, this design controls for the influence of such 

I potentially contaminating variables as History, Maturation, 

I Testing, Instrumentation, Regression, Selection, Mortality and 

I the Interaction of Selection and Maturation. 



I 



i Even though this design does not control for the possible 

I Influence that testing may have on the behavior, introducing 

I two additional control groups (the Solomon Four Group Design) 

I would excessively Increase the requirements of resources needed 

I to conduct such a study. In the task and pursuit of definitive 

I answers to questions regarding what caused what experimentally, 

I such factors as participants' frequent fear of "manipulation", 

I the employment of controls, the quantification of measures, 

the apparent omnipresent contamination Involved in field research 
and development, and other major technical problems may catalyze, 
mask or skew the findings. 

I In order to more accurately assess the effects of training and 

f control for alternative explanations of the findings, essentially 

the Pre-Test Post-Test Control Group Design might be usefully 
I Incorporated in a design for a third pilot test. 

I 6. The relative insuffioienoy of the sample and the heterocfeneity 

I of the trainincj group 

That as sample size Increases so does the chance that the sample 
statistics will approximate the population parameters is a basic 

! statistical premise. With an n=14, one hesitates to generalize 

the findings of this training program to the population of 
individuals for whom the program was designed. 
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Whether this sample represented the population further complicates 
the issue of sample size. Occupationally, the group of 14 



participants was heterogeneous: curriculum consultant, secondary 

teacher, guidance counselor, principal, superintendent. Title III 













Agency consultant, primary teacher, social studies consultant, 
special education teacher and learning specialist. Ordinarily, 
such a heterogeneous sample would be most acceptable. In this 
Instance, however, the fact that each occupation is represented 
by only one or two Individuals Inherently confounds the particular 
occupation with the Individual's personality. This contaminating 
factor generates the question, "Is the SCE's degree of success 
due to the Individual's unique personality patterns, his specific 
occupational background, the program, or an interaction of the 
three?" 

If — in the next pilot test — at least five individuals represented 
each occupation, contamination would be alleviated. Furthermore, 
not only would generalization to the population be less restricted 
but also the assumption about the program's applicability to 
different educational occupations could be tested. 
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Heterogeneity In participants' personality structures complicated 
the findings. In-depth Interviews with the training staff, some 
participants' comments, data derived from certain Instruments 
(Trainee Comparison Form, the IITRI Sociogram, the Participant 
Evaluation Exercise) Indicate that a few participants lacked 



Moreover, 



the desired fundamental qualities of a potential.' 
the staff and other participants perceived these individuals' 
Incompatibility with the program and unique behavior patterns 
as "hurting" the program. (That these few individuals were 
accepted for training can be attributed to the lack of 
sophisticated selection procedures available to CERLI at the 
time and extenuating circumstances precluding In-depth Inter- 
views with these applicants.) 



However "harmful" to the program these particular trainees may 
have been, they probably did provide a basis for assuming that 
the participants at Charleston represented the whole continuum 
of desirable and undesirable traits, qualities and behaviors that 
a potentially trainable SCE should manifest. 
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CONCLUSIONS 



Because of insufficient data and certain constraints, we 
neither can categorically accept nor reject the hypothesis 
that the training program effected behavioral changes enabling 
the participants to effectively function as Specialists in 
Continuing Education. 

However, we can confidently assert that available data do 
provide partial answers to the questions posed in the 
Introduction to this Assessment of the Cerli Training Program; 

Did the program do what it intended to do? 

Did the beha:oior of the participants change? 

If sOf in what direction? 

The results of the Evaluation Exercises suggest that the staff 
and the participants were generally satisfied with the program 
and believed that it had realized its objectives. 

The results of the Problem Approach Test and the Participant 
Change Survey suggest that trainees' behavior did change in 
the desired direction. 
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Though serious Issues clouded the analyses » one should not 
overlook the fact that many research and development projects — 
in their early stages-~encounter similar problems involving 
methodology • instrumentation* follow-up evaluation, criteria 
and sample size. 

The fact that CERLI staff members — who contributed significant ^ 
ideas to the assessment of evaluation constraints — perceive 
and endeavor to resolve these problems substantiates favorable 
progress in the program's development. 

Given this awareness of Issues and the vigorous attempt to 
resolve the problems, a third pilot test should result in 
significant and substantial progress toward consummating the 
development of the SCE program. 
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Appendix A; Description of Program of Instruction (POI) 
(E yerpted from Preliminary Ve r sion of the Basic Program Plan) 



B. Content 



The attitudes, knowledge, and skills to be communicated were organized 

into seven instructional units which were distributed throughout the 
six-week program. ® 



1. Orientation 



The purpose is to give a general explanation of what the program will 
entail so that participants will know what to expect, to acquaint 
staff with participants so that they may formulate balanced work and 
sensitivity teams, and to facilitate the entry of participants into 
team activities. 



a. 

b. 

c. 

d. 

e. 



f, 

g< 



h, 

i, 

j, 



Identify staff 

Identify each other 

Overview of program and schedule 

Purpose of training agency 

Purpose of developmental agency (will be deleted after the 
work of the developmental agency is completed) 

Team formation 

Explanation of feedback device 
Explanation of content options 

Acceptance of individual responsibility to the team 
Acceptance of individual responsibility to team activities 



Goal establishment methodology 



^is refers to the process of stating expected learner outcomes in 
behavioral terms. Goal establishment methodology is based on the 
assumption that it is sound practice to know what the learning process 
should achieve before commending the actual process. 



a. 

b. 

c. 



d. 



Recognize and identify a behaviorally stated goal 
State behavioral goals for his own operating situation 
Recognize, identify, and use systems which are valid and 
relevant for ascertaining stated goals 
Instruct others in operationalizing goals 
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3. Data collection methodology 



This refers to the operation of securing objective information about a 
particular behavior. By using data collection and interpretation, the 
SCE helps the professional assess his effectiveness in relation to his 
stated goals. 



a. 

b. 



c. 

d. 

e. 

f. 



Understand designing of research problems 

Familiarity with evaluation Instruments (identifying components 
of each, comparing instruments, and selecting appropriate 
Instruments) 

Design new Instruments as required 

Teach data collection and instrument development 

Willingness to use data collection methodology 

Why and how to use equipment for collection of data 



4. Confrontation 



This refers to self-encounter which brings one face-to-face with an 
internalized goal-data relationship. There is often a difference 
between what a teacher wants to do and what he does. Whenever a teacher 
develops operational goals and receives reliable data concerning the 
attainment of those goals, he may perceive a similarity or congruence 
between the goal (ideal) and the data (actual) , or he may perceive a 
conflict or discrepancy. In the seminar, the SCE must create a supportive 
atmosphere where this self-confrontation and the setting of possible 
new professional goals or behaviors can occur. 



a. 

b. 

c. 

d. 

e. 



Encourages continuous selfyeyaluatlon 
Confrontation theory and dissonance theory 
Utilize group resources in support of self— confrontation 
Constructive attitude — the helping relationship 
Collaborative rather than authoritative attitude 



5. Resource methodology 



Resource methodology is used by the SCE to provide the professional with 
substantive Input for new behavior. 



a. 

b. 

c. 



d. 

e. 



Acceptance of "search area" concept 

Non-authoritarian presentation of resources in seminar 
Locate, secure, adapt, and make available relevant material 
and human resources and have professionals do the same. 
Awareness of a resource knowledge base 
Understand search area concept 
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6. Group functionin g 

According to Cartwright and Zander, "A group is a collection of 
individuals who have relations to one another that make them inter- 
dependent to some degree". The group is the vehicle by means of 
which all of the SCE’s activities provide the environmental support 
system conducive to all other Interaction. 

a. Have the attitude that working and interacting with people, 

especially in groups, is rewarding work. 

b. Possess the attitude that collective strength is generally 

greater than individual strength in creating change within 
the individual. 

c. Possess the attitude that Individuals in the group will 

benefit from group interaction and the SCE will actively 
work toward that goal. 

d. Possess the attitude that professional growth is possible 

and feasible and more lasting through group interaction. 

e. Understand the origins of groups, the nature of group 

membership, structural properties of groups, communication 
patterns within groups, motivational processes in groups, 
and leadership functions and styles within groups. 

f. Have the skills and abilities to apply the above knowledge 

to the seminar. 



7. Administrative methodology 

This refers to the totality of skills required to implement and maintain 
functional inservice seminar programs within an educational setting. 

The objectfve is to provide the managerial framework within which the 

SCE will gain entry to an educational system and implement his inservice 
program. 



a. Gaining entry to a school system and securing administrative 

support (understanding and acceptance) for inservice 
program 

b. Scheduling of time, space, material, and personnel 

c. Gaining of community acceptance 

d. Recruitment of professionals for the progr am 

e. Maintaining active and open communication with training 

agencies and other SCEs 



f. Support CERLI in development of the training system (will 
be deleted after the work of the developmental agency is 
completed) . 
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Dorwin Cartwright & Alvin Zander, Group Dynamics: Research and Theory . 

Harper and Row Publishers, New York, 1968, p. 46. 
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C. Typical dally schedule 



The sequence of learning experiences was organized into a dally schedule 
which contained an outline of activities for each day during the six- 
week training program. A typical daily schedule would be as follows: 



8:00 - 


9:45 


d • in • 


Teaching, planning, search area 


9:45 - 


10:00 


a«m« 


Coffee break 


10:00 - 


12:00 


a • in • 


Seminar and critique of seminar 








by staff member 


12:00 - 


1:00 


p*in« 


Lunch 


1:00 - 


1:20 


p • m • 


Data Collection 


1:30 - 


3:00 


p*m* 


General Session 


3:00 - 


5:00 


p*in* 


Search area 


5:00 - 


7:00 


p • m • 


Dinner 


7:00 - 


9:30 


p ein* 


T-Groups 
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App6ndlx Bi Staff Evaluation Exarclsa and Sununary Data 



Staff Evaluation 
CERLI Leader of Continuing 
Charleston, Illinois 



Exercise On |he 
Education Training Program 
August 8, 1968 



Instructions for Completing The Questions 



This form Is designed to receive your opinions, judgments, 
and suggestions about the Pilot test just completed In 
Charleston. This Is not a test, there are no right or 
wrong answers. Please give your true reactions to your 
experiences during this program so that future programs 
will get the benefit of your Ideas and suggestions. 



How sati sfied were you with the advance Information provided about ; 

Not Satisfied Satisfied 

this CERLI program? 5 

Charles ton-Mat toon, Illinois? 5 



Eastern Illinois University? 
lodgin g ? 



meals 



other (specify) school conditions 2_ 

students ^ 

participants 1 



What Information would have been helpful ? 

Knowledge of participant selection criteria (2) 

Knowledge of participants’ background, experience and needs (2) 

More advance planning of locations of rooms, materials, etc. In school 
Alternate arrangements to meal tickets 















What Information should have been given on the first day that would have 
been helpful later on In the program ? 



Goals of program — ultimate behavior participants could expect to display 
as result of six weeks 



Goals of program — what is a seminar; program is a pilot test; program trains 
leaders — not goal establishing, data collection specialists 



Availability of rooms and space not in use 



What, in your opinion, are three Important obiectlve(s) of this CERLI 
program ? 



To help trainees try new behavior, to demonstrate that an individual can 
operate productively in groups (3) 



To pilot test a program of instruction (2) 



To train LCE in order to evaluate the POI (2) 



To train LCE (2) 



To help trainees learn process and theory of the helping relationship (help 
others assess their needs) (2) 



To help participants examine effects of their value system on their ability 
to function objectively in helping role 



To determine whether the program could be conducted successfully by staff 
other than the "Appleton" staff 



ASSESSOR'S scheme for interpreting summary data 
in questions 5,6,7,8,10,12,14,16,18,20,22,24,28, 
29,30,45,55: 

(mean rating in millimeters) 89.00 
(standard deviation in mm) 13.04 
scaled distance ° 126mm 



0«-» polnt of measurement begins at negative 
end of continuum 



How well did the program achieve the objective you listed jn 4a ? 



Think of the horizontal line below as a measuring rod 
or scale running from "completely achieved" to "completely 
not achieved". Mark the line at that point which best in- 
dicates how well the program achieved this objective. 






89.00 



13.04 



completely 

achieved 



completely 
not achieved 
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6 . 



How well did the program achieve the objective you listed under 4b ? 



D-^ 



103.20 



f 



completely 
not achieved 



H 



7.73 



completely 

achieved 



7. 



ILqw well did the program achieve the objective you listed In 4c ? 



f 



completely 

achieved 






112.40 



g 5A 



completely 
not achieved 



- 41A how w ell do you think the program has achieved Its objectives * 






completely 

achieved 



^0 

H 



107.00 



8.76 



completely 

failed 



9. 



Name three personal qualities you believe are most Important to effective 
leadership. 



Acceptance/respect for Individuality of others (3) 
Good listener (2) 



Self-awareness/Inner stability, flexibility 
Sense of humor 



Sensitivity to Individuals 



Ability to bring out Ideas of others 



Desire to help Individuals grow through self-designed learning 
Like people 



10 . 



Sensitivity to Individual expressions of needs 

How satisfied were you with the practice teaching components of the program ? 

87.60 



^ 0 



completely 

satisfied 



completely 

dissatisfied 



22.87 
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How wo uld you make the practice teaching more effective ? 
Make the situation more "real" 



Keep pupils 2/3rds. of workshop. Have closer grade groups/classes larger 
than eight 

Make pupil attendance rules and give certificates at close of session 

Teachers should spend less than two hours In the situation 

Have trainees work with summer/student teachers with real problems to 
eliminate trainees' preoccupation with teaching 

How satisfi ed were you with the practice leader setnlnars ? 

0 -> 



How c an the practice leader seminars be made more effective ? 

Have more "realistic" role playing — allow for variety of situations and 
persons (3) 

Make seminars less mechanical — more real: more emphasis on process — less 

on mechanical devices (3) 

Have seminars In afternoon so there will be more time 
Switch seminars 
Change groups ' 

Have work with teachers trainees didn't know 

Give trainees Individual problem-solving exercises when they have mastered 
confrontation and goal setting techniques 

jiow satisfied were you with the staff critiques of the practice leader 
seminars ? 



88.20 

34.78 



completely 

dissatisfied 



completely 

satisfied 



104.00 

14.85 



completely 

satisfied 



completely 

dissatisfied 
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15. How can the staff techniques be made more effective? 



r 



i 



16. 



17. 



I 



18. 



Rotate staff to only one presentation to prevent trainees from 
"latching on" (2) 

Specify topics for staff members to cover by analyzing tapes and 
listing areas already covered 

Allow staff time to listen to tapes of their critiques 
Compare staff roles by tapes 

Staff should pose problems to group — get specific Input 
Emphasize leader role more: teacher less 



How s atisfied were you with the lecture and general sessions ? 

o-» 



completely 

dissatisfied 



I 93.80 

» 7.73 

completely 
satisfied 



How can the lectures and general sessions be made more effective ? 

Advise "consultants" on problems/tone of group so Input can be specific 
and relevant (2) 



Orient guest speakers (2) 

Shorten lectures — have demonstrations, charts, recordings, etc 

Allow each resident staff to contrlbutje 

More participant Involvement 

More assessment of group needs 

Read participants ' reactions 

Give sessions In morning, before seminars 

How satisfied were you with the search area ? 



( 2 ) 






completely 

satisfied 



99.20 

19.21 



completely 

dissatisfied 
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19 • How can the search area be made more effective ? 

Improve physical environment — air conditioning! (5) 

Have separate clerical and equipment for area to lessen distraction 
and congestion 

Distribute complete list of materials to trainees 
Have more material 

Greater stress on participants contributing things to search area 
Analyze use of materials and questionnaire 
Allow more free time for trainees to use area 
Better use of consultants 

20. How satisfied were you with the evening small group sessions ? 

0 —^ 



completely 

dissatisfied 

21. How satisfied were you with your housing accommodations? 



Have a space for staff meetings 



108.80 

9.13 



completely 

satisfied 



<-P 



completely 

dissatisfied 



completely 

satisfied 

22. How can the group sessions be made more effective ? 

Remove participants with severe emotional problems (3) 
Have a more comfortable location (2) 

Trainees serve as check and balance (2) 

23. How can the housing be Improved ? 

Co-ed housing — would facilitate work sessions (4) 

No night hours for adults (3) 

Separate housing for staff, for students, for trainees 

Separate adults from students 

Have a central place for Informal activities 



74.20 

47.73 
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24. How satisfied were you with the formal and informal social activities? 





1 


1 102.00 




— 1 


1 19.81 



completely completely 

dissatisfied satisfied 

25. Ho\.r“ can the social activities be made more effective ? 

Have meeting place close to other activities (3) 



Have a place with fewer outside people and more "inside” people 

More formal — to Involve all participants 

Greater commitment from some participants 

Serve drinks 

More places to go 

26. If CERLI conducts another leader training program of the same total length of 
time (i.e. 6 weeks), would you recommend that each listed activity be given 
about the same amount of time as this year, or that it be increased or reduced 
in time? 



(Please check one in each line, and remember that the total time is fixed). 
\alue: 1 2 3 







■ Uw • .1. ^ 

I recommend: 




Activity 


More 

time 

+ 


About same 
as this year 


Less 

time 


Mean : 


a. 


practice teaching 






5 


3.00 


b. 


leader practice 


3 


1 


1 


1.60 


c. 


leader seminars 


3 


1 


1 


1.60 


d. 


staff critique of leader 
seminars 


1 


3 


1 


2.00 


e. 


general lecture sessions 




1 


4 


2.80 


f. 


search area period 


3 


2 




1.40 


8- 


evening groups 


1 


4 




1.80 


h. 


social mixing 


2 


3 




1.60 


i. 


free time 


5 






1.00 



B-7 



er|c 




























27. Please rate the coverage of substantive content In 



I 



o 
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I: 


II; 


III! 


1 " too advanced 


I too general 


A = Insufficient amount 


2 B about right 


II s about right 


B « about right 


3 » too simple 


III a too detailed 


C = disproportionately 






large amount 



a. 


goal establishment 
methodology 


1 


2 


3 


I 


II 


III 


A 


B 


m 

c 


b. 


data collection 
methodology 


2 


3 


1 


II 


III 


I 


B 


C 


A 


c. 


resource methodology 


3 


1 


2 


III 


I 


II 


C 


A 


B 


d. 


small group processes 


1 


2 


3 


I 


II 


III 


A 


B 


C 


e. 


confrontation techniques 


2 


3 


1 


II 


III 


I 


B 


C 


A 


f. 


group functioning 


3 


1 


2 


III 


I 


II 


C 


A 


B 


g- 


administrative methodology 


1 


2 


3 


I 


II 


III 


A 


B 


C 





*^Mean’‘ ratings: 


I: 


m 


III: 


"Mean 

of 

Means": 


a. 


goal establishment 
methodology 


2.00 


2.20 


1.60 


2.07 


b. 


data collection 
methodology 


2.60 


1.20 


1.40 


1.73 


c. 


resource methodology 


2.00 


1.80 


1.60 


1.80 


d. 


small group processes 


2.20 


1.80 


2.00 


2.00 


e. 


confrontation techniques 


2.20 


1.80 


1.40 


1.80 


f. 


group functioning 


2.20 


1.60 


1.60 


1.80 


g- 


administrative methodology 


2.00 


1.80 


1.40 


1.73 




"Mean of Means": 


2.17 


1.74 


1.63 
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28 . 



Please rate your overall satisfaction with the substantive content of 
the program . 

I 91.20 



extremely unsatisfactory, 
couldn't be worse 



I 7.41 

extremely satisfactory, 
couldn't be better 



29. How much exchange of Information and concepts - that Is . cross- 
fertilization of Ideas - took place among your fellow members of 
the program ? 

0-> 



minimum 

amount of exchange 
(practically none) 



I 102.00 

I 12.79 
maximum 

amount of exchange 
(completely overwhelming) 



30 . ^Ink now of the balance achieved by the program In meeting specific 
Individual needs as compared to meeting the needs of the group In 
general. How would you judge the balance achieved by the program i:a 
meeting these different needs ? 

^0 



balance was excellent 
couldn't have been better 



98.40 



9.26 

balance was poor 
couldn't have been worse 



31* What should be done to Improve this balance between Individual and 
group needs ? 

More Informal mixing with participants (3) 

Some demanded too much individual attention/screen out these people (2) 
Too much time spent writing POI during sesslon(2) 

Greater schedule flexibility and sufficient staff to allow for 
meeting individual needs without sacrificing group needs (2) 

Emphasize early, the importance of the group 

Weekly participant — staff interviews to determine participant needs 
Better and more communication between visitors and boondockers, 
especially prior to individual/ general sessions 
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32. - 43. The following questions refer to how adequately the program was 
administered by the staff. Circle the letter following each 
question t o indicate rating of adequacy using th© following code : 

A = extremely adequate; better than expected 
B » adequate; about as expected 
C = adequate but less than expected 
D » Inadequate; much less than expected 



32. 


practice teaching arrangements 




A 


B 


C 


D 


33. 


beginning sessions on time 




D 


C 


B 


A 


34. 


carrying out sessions as 


scheduled 


C 


A 


D 


B 


35. 


changing schedule to meet 


group needs 


B 


D 


A 


C 


36. 


providing advance Information on program changes 


A 


B 


C 


D 


37. 


maintaining objective relationships with 












group members 






B 


D 


A 


C 


38. 


helping group members with their special 












program needs 






C 


A 


D 


B 


39. 


helping group members with their soeclal 












personal needs 






D 


C 


B 


A 


40. 


providing requested resources or facilities 


A 


B 


C 


D 


41. 


social mixing with group 


members 


B 


D 


A 


C 


42. 


resolving conflicts among 


group members 


C 


A 


D 


B 


43. 


other (specify) 






D 


C 


B 


. A 




FREQUENCY Tallv: 














32. 


A=0; B=3; C=2; D=0 


Mean 


2.40 










33. 


D»0; C=0; B=l; A=4 


Mean 


1.20 










34. 


C*P; A=3; D=0; B=2 


Mean 


1.40 










35. 


B=2; D=0; A=3; C=0 


Mean 


1.40 










36. 


A* 5; B=0; C=0; D=0 


Mean 


?Sfl.OO 










37. 


B=2; D=0; A* 2; C=1 


Mean 


- 1.80 










38. 


C = 0; A = 3; D = 0; B « 2 


Mean 


1.40 










39. 


D*0; C=0; B=2; A=3 


Mean 


1.40 










40. 


A* 2; B = 3; C = 0; D = 0 


Mean 


1.60 










41. 


3=2; D=l; A=0; C=1 


Mean 


2.20 










42. 


C=0; A=2; D=0; B=2 


Mean 


1.50 










43. 


D«0; C = 0; B = 0; A=0 





— 
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space for additional comments about the administration of the program 
on the CERLI staff : 

Satisfied with Charleston staff; CERLI staff in 
administering program; CERLI staff solidarity 
and commitment 

All in all, please indicate how you would rate the administration of 
the program . 

I 1 12.20 

1 I 9.41 

extremely adequate, extremely inadequate, 

better than expected much less than expected 

Looking back over the entire program, what specific part s or things, 
were most effective ? 

Seminar of role practice (2) 

Pre-planning and scheduling of activities by CERLI prior to Charleston 
Sensitivity training — individuals trying new behavior 
Seminars and search area 
Seminars and T-groups 

Unity of staff toward trainees i.e. similarity of attitude, procedure, 
general morale 

What specific parts or things were least effective ? 

General sessions (3) 

Shallow seminars — second half of session 

Women— staff --women participants conferences: trainees preferred male help 

Interviews with additional resources 

What kinds of activities (lectures, discussions, substantive contejit , ^ 
se"<^nars% small groupis, etc.) should be eliminated from the program, 
if it is given again? Please give your reasons . 

Lectures should be shortened greatly or eliminated if tney compose the 
major substantive input source and if lecturer isn’t oriented to 
background and tempo of group. (2) 

More flexible schedule — should be based on group needs 
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49 . 



Extra-curricular demands on staff because of POI deprived participants 
of needed consultation time. 



What kinds of activities not Includod in the program should be added ? 
Please give your reasons . 



Real teachers to conduct seminars and give "real" situations, thus 
avoiding the "mechanical" seminar (2) 



More explanation of the program 



Clearer presentation of interaction analysis systems so trainees could 
move toward more complex goals 



Have more specific group exercises i.e. inclusion — exclusion operations 
to enable all participants the opportunity to feel these experiences 



50. 



- 53. (Values assigned to choices in the following questions indicated 
in parentheses.) 



50. 



Was the size of the group in the program too large, too small, or 
about right ? 



51. 



(3) too large (1) t oo small (2) a bout right/Mean; 1.00 
How would you rate the size of the leader practice seminar groups ? 




I 



I 



I 








Ideally, what follow-up activities are needed from CERLI to help the 
leaders succeed in imp lemea ting and then administering inservlce leader 
programs in their schools ? 

Two-day follow-up sessions (3) 

Monthly communications — newsletter (3) 

Series of (whole or part) group conferences to compare problems 
and solutions (2) 

CERLI staff visits to make suggestions (staff critique) (2) 

Phone calls/visits to trouble-shoot any problems (2) 

CERLI staff visits to observe LCEs operating 

Data collection on effectiveness and difficulties of LCEs 

Action by CERLI when asked for specific help 

Basic support plan as outlined by participants 

55. Mark the line below at the point which best indicates the degre e of your 
satisfaction with the program as a whole . 



0 — ^ 



very unsatisfactory 



I 103.00 
I 8.29 
very satisfactory 



(l)Thls was a very rewarding experience personally. I enjoyed the work with 
groups in the seminars particularly, and with the staff as a group. I would 
have been happier with a larger group; for example 24 people. More time 
to evaluate the program as it was operating would have helped, but that was 
hampered by writing the POI while there in Charleston. I reacted negatively 
to Boondock Staff title — probably because it came from outside the "Boondock" 
Staff--whlch again proves how closely that staff did work. I was very 
pleased with the fact that the participants became a group and that O.E. 
seemed affirmative about what they saw in Charleston. I was also pleased 
that we did complete the rewrite of the POI while there. 

b43i 
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( 2 ) 



On a personal level I enjoyed the six weeks. I felt I grew as a pro- i 

I 

fessional as did the other 5 members of the staff team. 

I felt the 6 weeks at Charleston was much better organized than Appleton. | 

I think we will be able to determine some of the variables and what 

happened. This will enable us as a staff to objectively evaluate the j 

various components of the program and it’s effect. | 

I was concerned with a somewhat superficial grasp of the leader role many 
of the participants possessed. It seemed too mechanical. I feel this 



could be improved by some real situations. 

I saw some personal growth on the parts of at least 3/4 of the participants. 
It will be interesting to note if some of this new behavior continues. 






J 




(3) I'm glad it's over! I felt burdened most of the time because of the double 
task of working with participants and rewriting the POI. The tight schedule 
did not allow for attention to my personal needs. I often felt unkempt by 
the end of the day. 

I hated the third rate motel in which we met and worked. The rooms were 
uncomfortable at the dorm, in the lab school and the motel. 

The program would have been improved if the resident staff had met for a 
week's pre-planning. 

I feel good about Pat and his leadership and also I'm Impressed with the 
capabilities of the resident staff. 



I- 

I (4) 



Opportunity for personal growth in an extremely supportive environment. 
Development of an ideal "Text-Book" team, and participation in the 




development of that team. Participation in the "Helping Relationship" with 
participants. Overall an extremely rewarding experience — like being un- 
fettered for the first time. 
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CERLI Leader of Continuing Education Training Program 

Charleston, Illinois August 8, 1968 

Instructions for Completing The Questions 

This form is designed to receive your opinions, judgments, 
and suggestions about the training you have just completed. 
This is not a test , there are no right or wrong answers. 

We are not looking for unearned "pats on the back" or 
unearned "slaps in the face"; we simply want to know your 
true reactions to your experiences during this program so 
that future programs will get the benefit of your ideas 
and suggestions o 



How did you first learn about the training program you have just completed ? 
6 from another CERLI-trained leader 

1 from my supervisor (department head, principal, superintendent, etc.) 

0 from a teacher or other colleague 

0 f rom a personal friend 

2 from a staff member of CERLI 

__ 5 __ o ther (specify): Title III project director; EDISU-Richardson, Cole; 

CERLI information; EPIC staff-Ed Brown; EIDSU; letters about CERLI 

summer institute distributed in x’s school 

What was your first reaction when you learned that you would definitely 
attend this program ? 

Pleased (5); Opportunity to learn new approaches to administration, leader- 
ship (3); Pleased but anxious about own qualifications; Pleased and curious; 
Interested and anticipated profitable experiment; Teaching; Proud; Pleased 
but questioned program validity 
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3 . 



What proerams did you consider before deciding on this CERLI program ? 
6 none 



0 EPIC 



2 E DSU program in Charleston 

3 _N.E.A. Institute (specify) Title ; 

NDEA-European History Seminar (Georgetown) 

Teaching of the gifted: Social Studies & Science (Lockport, 111.) 

Human Relations Laboratory I (Bethe^l» Maine) 

2 U niversity summer course(s) (specify): 

Course (s): Flexible Scheduling 

Education Courses 

University: Northern Illinois University 

Other (specify) : 

Northwest Multi-Service Training Center (Valparaiso) 

Space Workshop; non-credit reading workshop 

Please rank the Importance of each factor below in helping you decide to 
attend this CERLI program, as opposed to other summer programs. 

Let "l'° r ank the most Important, ”2" rank the next most Important, and so on . 

Rank Factor 

Mean: 



8.55 the only program available 
6.67 n o tuition charge 

7.27 l ocated in Charleston 

2.00 p otential for my professional advancement 
4.08 m eans to enhance my position and authority 

4.83 a ttendance will guarantee continuistg contact with CERLI 

6.27 m eans for salary Increase in my present job 

6.00 o pportunity for future high-salaried job 

1.77_ d esire to actively participate in solving problems of education 

o ther: Provide Inservlce training program in x's school; x's supt. 

asked x to attend and strongly encouraged same; personal challenge: 
desire to acquire tools, skills, confidence for leader role; learning 
about educators and innovative educational methods; working with inter- 
cultural history and Carver students 
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Did you prepare yourself in any special wav for this CERLI program ? 
Check all the following that apply * 

Frequency tally ; 

number response 

of 

respondents 



5 




Read 1 or more books: 



12 

1 

1 



0 

6 

1 



Read 1 or more 






iournal articles: 


12 


0 




1 


3 




1 


1 


Took a special course: 


12 


0 




2 


1 


Other : 


10 


0 




4 


1 



Did not prepare In 
any special way for 

this program; 10 0 

(4 DID prepare) 

6. How satisfied were you with the advance information 
provided about : 

Frequency tally : 



not 

satisfied satisfied 

this CERLI program? 6 8 

Charleston-Mat toon, 111.? 3 11 

Eastern Illinois University? 3 10 

lodging? 2 11 

meals? 1 11 

other? (specify) 3 0 
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What information would have been helpful ? 





More specific information about finances, credit, meals, housing— also 



off campus (3) 



Information on search area material 

Information should h a ve been given on the first dav that would 
Men helpful later on in the program ? ~ 

No other information (7) 

cedi's objectives— overview and thorough explanation of progrs . (3) 
Orientation to equipment 
Introduction of participants 

Advise participants of uneasiness of structureless society 
Emphasis on growth concept — not pass — fail 



T-groups: create leaders aware of /sensitive to their perception of ( 

and by others. Able to work with diverse personalities (10) 

To train teachers to state terminal objectives, classroom goals in 
measurable terms (5) ® 

To develop leadership skills (5) 

To continue/ follow-up the leader program (4) 

To orient leaders to innovations in education (4) 

To stimulate awareness of group process (3) 

To facilitate adaptability of participants (3) 

To collect CERLI data/research data 
To improve student behavior 
To develop effective communication 
To develop listening habits 
To promote self-confrontation 

How well did the program ac hieve the objective you listed in 9a.? 




completely not achieved”. Mark the line at that 
point which best indicates how well the program 
achieved this objective. 



Think of the horizontal line below as a measuring 
rod or scale running from "completely achieved" to 



It 



completely 

achieved 




29.77 



completely 
not achieved 
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13. 



14. 



15. 



16. 
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11 • How well did the program achieve the objective you listed under 9b . ? 
0 ' 



completely 
not achieved 



73.07 



40.91 



completely 

achieved 



12. How well did the program achieve the oblectlve you listed In 9c .? 



1 


C 


93.50 


1 




32.47 


completely 


completely 


achieved 


not achieved 



How well do you think the program has enabled you to help teachers 
select valid Instructional goals ? 






completely 

unable 



86.07 



26.88 



expertly 

able 



How well do you think the program has enabled you to help teachers 
define their goals In operational terms ? 



expertly 

able 



i 



82.86 



37.27 
completely 
unable 



How well do you think the program has enabled you to help teachers 
select appropriate methods of collecting data for determining success 
In achieving their goals ? 

F5 



85.00 



32.05 



completely 

unable 



expertly 

able 



How well do you think the program has enabled you to help teachers 
find appropriate resources for their problems ? 

— 0 



83.14 



37.01 



expertly 

able 



completely 

unable 
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17 . 



18. 



20 . 
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How well do you think the program has enabled you to conduct leader 
s eminars with small groups of teachers ? 

g: ' 



87.21 



completely 

unable 



33.20 



expertly 

able 



How well do you think the program has enabled you to understand and 
use confrontation techniques ? 



H 



84.07 



expertly 

able 



37.26 
completely 
unable 



19 . How well do you think the program has enabled you to Implement and 
administer Inservlce leader seminars In a school ? 



77.07 



completely 

unable 



38.03 



expertly 

able 



All In all, how well do you think the program has achieved Its objectives ? 

■0 

95.71 



completely 

achieved 



I 28.18 

completely 

failed 
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21. Indicate the degree of confidence you have In your ability to perform each 
of the following by circling the appropriate code number In each line . 



Code 



Extremely confident, completely sure of ability 
Highly confident, one or two weaknesses In ability 
Confident, good general ability with some weaknesses 
Mildly confident, many weaknesses In ability 
Not confident, completely unsure of ability 



to subordinate self-interest 
for the sake of group 
Interest 



to make decisions 



m. to make others follow rules 
against their will* 



n. to understand others 
to make friends 

P* 



to be sympathetic to the 
problems of others 



5 4 3 2 

(*These behaviors may be regarded as undesirable In an SCE) 
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Mean: 



a. 


to 


communicate Ideas 


1 


2 


3 


4 


5 


2.14 


b. 


to 


tell others what to do* 


5 


4 


3 


2 


1 


2.57 


c. 


to 


understand new concepts 


1 


2 


3 


4 


5 


2. A3 


d. 


to 


adapt to new situations 


5 


4 


3 


2 


1 


1.71 


e. 


to 


make people follow order* 


1 


2 


3 


4 


5 


3.14 


f. 


to 


work well with others 


5 


4 


3 


2 


1 


1.71 


g- 


to 


organize others 


1 


2 


3 


4 


5 


2.43 


h. 


to 


lead others authoritatively* 


5 


4 


3 


2 


1 


3.00 


1. 


to resist the Influence of 
others* 


1 


2 


3 


4 


5 


2.79 


j. 


to view problems in a broad 
sense 


5 


4 


3 


2 


1 


1.93 



1.96 












24, 



From thG abovG list (ouGstion 21) , pick the three quail tiGs you 
believe are most Important to effective leadership . (Enter the 
letter in front of the selected quality In question 21) . 



In responding to 


6 selected a 


to 


1 selected b 


to 


1 selected c 


to 


5 selected d 


to 


1 selected e 


to 


5 selected f 


to 


1 selected g 


to 


1 selected h 


to 


1 selected 1 


to 


4 selected j 


to 


4 selected k 


to 


1 selected 1: 


to 


0 selected m: 


to 


4 selected n: 


to 


0 selected o; 


to 


7 selected p; 


to 



sake of group Interest 



(NOTE; only one selected Items b,e,h, and 1 respectively; none 

selected m the behaviors considered undesirable in an SCE.) 



23. How satisfi ed were you with the practice teaching components of the prograTn? 





^ — ( 


) 

63.21 






29.93 



satislfed 



completely 

dissatisfied 



If you were running the program, how would you make the 
practice teaching more effective ? 



Make definite rules about class time, punctuality, 
attendance, completion of assignments (8) 

Have more students (3) 

Have more materials to cover student ability range (2) 

Better dorm supervision so kids are punctual 

Emphasize that teaching should relate to world outside classroom 
Encourage student-teacher interaction 
Not emphasize only the Harvard series 
Teach one week at a time 

Have program set up; 2 weeks' — goal setting techniques 

2 weeks — other educational-improving techniques 
2 weeks — absorb search area materials 
Let participants suggest teaching- techniques improvements 
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25. Ho w sa^fled we re y ou wi th the practice leader 



27 



28, 



completely 

dissatisfied 



93.14 
24.77 
completely 
satisfied 



26. 



29, 



How can the practice leader s e minars be made more effective ? 

Rotate members of groups, staff members (6) 

More real/"gut level" feedback from participants and staff (3) 

Give more opportunities to play leader role (2) 

Clarify duties expected of a leader (2) 

Have more role-playing 

Improve presentation of confrontation-goal setting to 
make it less artificial ^ 

No change 

Present other seminar techniques besides confrontation, goal setting 

critiones of the practice leader 

0 -^ 

67.86 



h 



completely 

satisfied 



39.56 
completely 
dissatisfied 



How can the staff cri tiques be made more effprn„„ ? 

Satisfied (5) 

More staff rotation (2) 

De-emphasize staff role 

Staff members shouldn't have disagreed about concepts 
they were presenting 
Critiques too superficial 

I g SL- g^ati s fied were you with the lecture and general se.s^on. ■> 




completely 

dissatisfied 



J 67.86 
I 39.56 
completely 
satisfied 
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30. 



How can the lectures and general sessions be made more effective? 



More preparation by staff — better presentation, arrangement, 
organization (5) 

Deal with fewer topics in greater concentration (3) 

Waste less time joking, getting started (3) 

Start feedback earlier (2) 

Prefer discussion led by speaker over lecture (2) 

Use more outside resource people — Reynolds, Furbay 
More variety In materials 
More interesting materials 
Spread out written assignments 



32 . How can the search area be made more effective ? 

More comfortably equipped, quiet, less crowded study area (9) 

Organization to materials — files, annotated bibliography (3) 

More films available 

More time to use area 

More variety of materials 

Have a duplicating service 

Explanation of group dynamics device 



34. How can the group sessions be made more effective ? 

Have introduction/direction at beginning (2) 

Have outside trainer (2) 

Ease feeling of tension/ frustration in groups (2) 

Group not committed to honesty to sensitivity coordinate T-group 
activity in other activities 
Include staff members in T-group 
Have longer sessions 
Meetings too tiresome at end of day 
Satisfied 



31. How satisfied were you with the search area? 




34.89 



completely 

satisfied 



completely 

dissatisfied 



33. How satisfied were you with the evening small group sessions ? 
0 —^ 



93.25 

24.25 



completely 

dissatisfied 



completely 



satisfied 
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How satlslied were you with your housing accommodations? 



completely 

satisfied 



^ — 0 

I 81.60 
\ 51.75 
completely 
dissatisfied 



How can the housing be Improved ? 

House the group together, eat together (3) 
Satisfied (3) 

Not applicable — commuters (3) 

Don't be housed with pupils (2) 

Have low rates 

Need key to outside door 



How satisfied were you with the formal and Informal social activities’ 

Q — ^ ' 



completely 

dissatisfied 



I 90.58 

I 28.76 
completely 
satisfied 



How can the social activities be made more effective? 



Greater variety of activities— something besides drinking (4) 
Desire more togetherness and participation (3) 

Staff should participate along with trainees (3) 

Formally planned rather than impromptu activities 
Staff should not force its norms on participants 



How satisfied were you with the meals and meal 



completely 

satisfied 



arrangements 



0 



84.18 

36.57 



completely 

dissatisfied 



How can the meals and meal arrangements be Improved ? 

Have less expensive meals (3) 

Have better food (3) 

Make it clear that meal tickets were an option to cash payment (2) 
Satisfied 

Prefer cafeteria style 

Neal tlcicet too costly for amount of food one can eat 
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Next year if CERLI conducts another leader training program of the 
same total len.^th of time (i.e. 6 weeks) « would you recommend that 
each listed activity be given about the same amount of time as this 
year, or that it be increased or reduced in time? 



(Please check one in each line, and remember that the total time is 
fixed) . 
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I recommend: 




Activity 


More 


About same 


Less 








time 


as this year 


time 








+ 




— 


Mean: 


a. 


practice teaching 


2 


9 


3 


2.07 


b. 


leader practice 


11 


3 


0 


1.21 


c. 


leader seminars 


8 


6 


0 


1.43 


d. 


staff critique of leader 












seminars 


8 


5 


1 


1.50 


e. 


general lecture sessions 


1 


5 


7 


2.46 


f. 


search area period 


5 


6 


3 


1.86 


g- 


evening groups 


3 


7 


4 


2.07 


h. 


social mixing 


2 


11 


0 


1.85 


i. 


free time 


11 


3 


0 


1.21 




(Values = 


1 


2 


3) 








42 



Please rate the coverage of substantive content in 

the program as follows (circle one code in each column) : 



I: 



II: 



III: 



1 too advanced 

2 = about right 

3 = too simple 



I = too general 
II = about right 
III = too detailed 



A = Insufficient 
amount 

B = about right 
C = disproportionately 
large amount 



Topic : 

a. goal establishment 
methodology 

b. data collection 
methodology 

c. resource methodology 

d. small group processes 

e. confrontation techniques 

f. group functioning 

g. administrative methodology 



I: 



II: 



III: 



12 3 

2 3 1 

3 12 

12 3 

2 3 1 

3 12 

12 3 



II 



III 



II 



III 



II 



III 



III 



II 



III 



II 



II 



III 



II 



III 



ABC 

B C A 
CAB 
ABC 
B C A 
CAB 
ABC 



*'Mean" ratings : I: II: III : "MeSn of Means": 



a. goal establishment 
methodoloev 


2.29 


1.79 


^ ^ ^ • 
1.93 


2.00 


b. data collection 
methodoloev 


2.36 


1.64 


1.43 


1.81 


c. resource methodoloev 


2.08 


1.64 


1.64 


1.79 


d. small eroup processes 


2.23 


1.64 


1.93 


1.93 


e. confrontation techniques 


2.14 


2.00 


1.79 


1.97 


f. eroup functlonlne 


2.14 


1.79 


2.21 


2.04 


e* administrative methodoloev 


2.46 


1.36 


1.38 


1.73 




43 . Please rate your overall satisfaction with the substantive content of the 
program . 



.CtlU#K 



extremely unsatisfactory, 
couldn’t be worse 



80.43 



I 31.58 

extremely satisfactory, 
couldn't be better 



44 . How much exchange of Information and concepts - that Is. cross^'fertlllza- 





3 ) 




73.57 






32.74 



mlnlmum 
amount of exchange 
(practically none) 



maximum 

amount of exchange 
(completely ovewhelmlng) 



45. Think now of the balance achieved by the program In meeting your specific 
needs as compared to meeting the needs of the group In general. How 
would you judge the balance achieved by the program In meeting these 
different needs ? 

L 1 I 83.36 

r I I 29.68 



balance was excellent ~ 
couldn't have been better 



balance was poor - 
couldn't have been worse 



46. Was any person connected with the program Impolite, discourteous, or 
rude to you or to any member of the group ? 

8 no 



6 y es 

47. Space for additional comments on question 46: 

Better screening of participants — keep out problem participants (3) 
Have a course In human relations 



Situation not Intellectually stimulating (//16) made me tense and bored 
One staff member Ignored me (//14) One participant unable to function with 
reality and hurt many students and trainees 
//16 and Nick Louis were objectionable (//5) 


















48.-59. 



48. 

49. 

50. 

51. 

52. 

53. 



54. 



55. 



56. 

57. 

58. 

59. 



The following questions refer to how adequately the program was 
administered by the staff. Circle the letter following each 
question to indicate rating of adequacy using the following code ? 



A » extremely adequate; better than expected 
B « adequate; about as expected 
C * adequate but less than expected 
D s inadequate; much less than expected 



practice teaching arrangements 
beginning sessions on time 
carrying out sessions as scheduled 
changing schedule to meet group needs 
providing advance information on program changes 
maintaining objective relationships with 
group members 

helping group members with their special 
program needs 

helping group members with their special 
personal needs 

providing requested resources or facilities 
social mixing with group members 
resolving conflicts among group members 
other (specify) 



A 

D 

C 

B 

A 



B 

C 

A 

D 

B 



C 

B 

D 

A 

C 



D 

A 

B 

C 

D 



B 



D 



D 



B 



D 

A 

B 

C 

D 



C 

B 

D 

A 

C 



B 

C 

A 

D 

B 



A 

D 

C 

B 

A 



FREQUENCY Tally: 



48. 


A 


S3 


1; 


B 


a 


5; 


C 


a 


2; 


D 


a 


6 


Mean 


m 


2.93 


49. 


D 


a 


2; 


C 


«t 




B 


a 


6; 


A 


a 


0 


Mean 


a 


2.71 


50. 


C 


a 


2; 


A 


a 


U 


D 


a 


2; 


B 


m 


9 


Mean 


a 


2.36 


51. 


B 


a 


6; 


D 


est 


0; 


A 


a 


7; 


C 


a 


1 


Mean 


a 


1.57 


52. 


A 


a 


3; 


B 


a 


9; 


C 


m 


2; 


D 


a 


0 


Mean 


a 


1.93 


53. 


B 


a 


5; 


D 


a 


1; 


A 


a 


6; 


C 


a 


2 


Mean 


a 


1.86 


54. 


C 


a 


1; 


A 


a 


8; 


D 


a 


2; 


B 


a 


3 


Mean 


a 


1.79 


55. 


D 


a 


2; 


C 


a 


1; 


B 


a 


1; 


A 


a 


10 


Mean 


a 


1.64 


56. 


A 


a 


6; 


B 


a 


7; 


C 


a 


1; 


D 


a 


0 


Mean 


a 


1.64 


57. 


B 


= 


6; 


D 


a 


0; 


A 


a 


5; 


C 


a 


3 


Mean 


a 


1.86 


58. 

59. 


C«2; A»3; D=*4; B«5 

Only two participants responded: 


Mean 

» 


m 


2.50 



"Professional 
"Being human 



assessment of participants" 
(A) 



(C) 
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Space fog addition al conunents abou t the administration of the 
program on the CERLI staff; ” 



Better organization of rooms, materials, etc. 

Give individual help to participants 
More definite structure to general sessions 
Staff was found to be honest and program enabled participant 
to find own effective teaching practice 

Satisfied with resident staff— visiting staff often lacking 
Satisfied 

Satisfied, but staff appeared condescending at first 



AL\ .^a, al lj pleas e ^indicate how y ou would rate the administration of 

the proggam a — 

0 

— — I 87.54 

I 31.80 

extremely inadequate, 
much less than expected 



extremely adequate, 
better than expected 



AkPUt how^much of what y ou learned in this program will you be able to 
■M§Jg..wk en you begin functioning as an inservice leader in a school ? 

Q . * 11 1 11 ^ 



none of it 



all of it 



98.14 

22.03 



ba c k_o ver t he entire program, what specific part or thing was 
of greatest interest to you ? 



T-groups (3) 

T-groups and seminars (2) 

Confrontation— goal setting in operational terms (2) 

Seminars 

T-groups and leadership role 

Programmed Instruction lecture and establishing of teaching goals in 
operational terms 

Objective behavioral measurement 
Oi/ercoming personality faults (//16) 

Involvement with students especially Carver students 

Presentation of resource references on group dynamics 

Approach of using teachers to train teachers to improve their behavior 




kinds of aefc ivities (lectures, discussions, substantive content , 
seminars, sm all group sessions, etc,) should be eliminated from the 
■P-ro.Rrsm i£ it Is given _again to another group similar to yours ? 

General session topics— concentrate on a few rather than skim many (4) 
Shorten evening sessions (#16) (2) 

Questionnaires 

No elimination— but more variety 
General sessions— many irrelevant, too long 
Hostility and apathy to program 
T-groups— waste of time 

^kat kinds o f activities should be added, that were not Included in 
the program you have lust completed ? 



More free time 

Librarian — person to help trainees use search area materials 

More time for smal?^ groups 

Lecture on human relations 

More time on techniques — like problem-solving 

Substitute oral tasks for questionnaire 

More about group process activity 

More on goal evaluation and measurement, constructing Instruments to measure 
Start group activity in course about support designs and search area 
designs earlier than last week of session 
Activities were illiterate— add something of aesthetic nature 

67.-71. (Values assigned to choices in the following questions 
indicated in parentheses.) 

Was the size of the group in the program too large, too small. 
or about rleht l* 

large (1) t oo small (2) a bout right /Mean: 1.86 

6®* How would you rate the size of the leader practice seminar groups ? 

J,tL.too large ID t oo small (2) a bout right /Mean: 1.79 

69. How about the total length of the program: do you feel It was too 

long, too short, or about right ? 

■jL3)..,..too long (1) t oo short (2) a bout right/Mean: 2.14 

70 . How has this program matched your expectations ? 

13) — better than expected (2) a bout what I expected (1) n ot as good 

as expected/Mean: 2,43 
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mm. 



-Do yo^ ex pert to imeilenienfc an inservice leader seminar program 
fog teaghega in your school? 

(^) y ea Cl) , n o/ Mean? 2«00 

schools » 80 CT 6 Issdegs h3V6 reported pgobleins 
in impleme nting and_admlni8tegi^ Inservice leader seminars. How 
nM ch.diffiauli£v, If any, do you expect with each of the following? 

Code 

1 extreme difflc®lty 

2 moderate difficulty 

3 minor difficulty 

4 little or no difficulty 



a. shortage of facilities (space, 
office supplies, etc.) 


1 


Circle 

2 


one: 

3 


4 


Mean: 

2.15 


b. shortage of funds 


4 


3 


2 


1 


1.92 


c. pressure of regular duties 


1 


2 


3 


4 


2.38 


do lack of support from management or 
officials 


4 


3 


2 


1 


3.08 


e. resistance to change by colleagues 


1 


2 


3 


4 


2.23 


f. lack of authority 


4 


3 


2 


1 


3.23 


g. inadequate training 


1 


2 


3 


4 


2.92 


other (specify) 
h» Informality Cl entry) 


4 


3 


2 


1 


2.00 


i. follow-up Cl entry) 


1 


2 


3 


4 


1.00 




4 


3 


2 


1 





-Meally , vhat follow-up activities do von expect from CERLI to help 
you succeed In i mplementing and then administering an inservice 
leader proggamf 



Letters, newsletter, brochures (4) 

Phone or mail consulting service (4) 

Visits (3) 

Regular general meetings /conferences (3) 

Expect all support designs the group perceived to be in operation (3) 
Phone calls (2) 

Expect to receive new information (2) 

None; but would like aid in self-evaluation in regard to participation 
in seminars 

Reading material lists, tapes 



74. 



Mark the line below at the point which best indicates the degree of your 
satisfaction with t he program as a whole 
0 — ^ 



very unsatisfactory 



92.57 



24.76 
very satisfactory 
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75 . Please wtice a shoirt summagy or your own personal reactions to this 
program* Mentaoia aa'vi.higig, whether or not covered in the questions 
above* you liked best and also the things you disliked or thought of 
little imporea'aee a 

...I got a lot out of the program in terms of really looking at myself 
closely and hav/’ing others make just and fair comirjents on my personal and 
professional behavior^ 

I am most impressed with the summer program because of the level of 
honesty that the staff maintained throughout the si2c-weeks. 

I feel that I have much more self-confidence and ambition to do what I 
can and to encourage others to change and improve education. 

I was able to con»e to grips with the disappointments and anxieties in 
my work and thus be confident enough to try new behavior and accept other 
dissimilar behavior* 



..»! especially enjoyed the -friendly attitude of the staff and their ability 
to place everyone on an equal level in the realm of acceptance. 

I shared a feeling In the early days of the workshop that Information and 
material were being made available to the participants in too small 
amounts; then the reverse near the end in the rush to get everything 
completed. I felt that earlier confrontation of the group by the objective 
would have provided more time to absorb and prepare the final design 
documents in a more presentable form with better understanding. This is 
the common falling of all of m as teachers or leaders. We tend to 
under— estimate the ability cf our students or participants to perform* 



. 0.1 found this to Ibe a very rewarding experience for perception into my 
behavior as seen by others and myself and the affect it has on others in 
relation to thfi; effect I am striving for. 

There was a very capable staff but it was over-burdened in many respects — 
pressures by home base staff, researchers, participants, personal situations 
outside the institute* They did their best and I’m very proud to have 
worked with each of them* 

I disliked the fact that screening of participants was not adequately met. 
This has been a trying experience for all. 



..ol feel that the program should be longer so that the role of the Leader 
could be strengthened. I realize that the best experience is in the 
field, but I am fearfui that some of the participants are going to run 
into diffic^rflties they can’t handle. This in turn reflects on CERLI 
and the entire idea of iContlnwing education. 

More work should be done liu the area of goal setting- — ability to state 
operational terms tfecrt (Can be measured. 

Last but not least — I wish you luck on selecting participants. 
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...six weeks seemed to be too much time to learn the leadership and confrontation 
techniques . 

The seminars at times were interesting; other times we were bored. 

(^e person always set the tone for the seminar: it was her way or nothing else. 
We could have done more role playing in the seminars which would have been a 
T general sessions on Programmed Instruction and Problem Solving. 

I dldn t like the meetings not starting on time. I didn't like the students’ 
attitude toward coming to class— they were late and several times didn't come. 



I 



r 

I 



...The approach to Improvement in teaching by objective measurement of teacher 
behavior and related student behavior is an appealing process. 

Teachers with problems and real desire to solve those problems will accept the 

group seminar process and see it as a resource in their school for helping 
teachers. 



...I think I have well covered everything of importance In the foregoing items 
of things I liked and disliked, I think, however, the business of being seen 

as others see you may be important to us everyone rather than "you but not me" 
attitudes. 



...I thought the goal setting techniques and procedure were good. These were 
not very well explained but were gradually picked up, I do wish we could have 
Interchanged with other groups, I also thought other Inservlce ideas could 
have been presented in much more depth, I was disappointed in this ex-forced 
field analysis, inquiry, group dynamics etc, A few of these In more depth 
would have helped. When some of these were presented in an afternoon session, 
the staff member, many times, seemed not to be prepared or have little bac^k- 
ground in the area. I believe six-weeks is a little too long for this program 
All in all I enjoyed the program and think I did receive benefits from it. 



,,,The program has reinforced some of my own thoughts as well as helping me 
to crystallize much of my own thinking. The emphasis on the questions: What 
do you want to do; why do you want to do it and how can you tell if you have 
done it or not was most appropriate. These seem to me to be crucial to 
education as well as to anything else, I like very much the corresponding 
emphasis on the fact that it is the teacher who must confront himself with 
these questions, that goal setting and the seminar approach are a process not 
a technique, and that as leaders we are supporters, not directors within that 
process. To the staff I would suggest in the beginning, less grouping of staff 
members and less bantering in the jargon that participants are not familiar 
with. Also, less emphasis^on partying .as It.^seems at times as If this, rather 
than the program, is one of ybur greatest conc^hs. 
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...The thing that aiKSized me the most was closeness of group (a couple of 
exceptions) and the concern of the group for one and another. I gained a 
lot of personal confidence in this experience and a better idea of how 
people feel about me. I have become more aware of the feelings and actions 
of other people. I have gained some life-long friends. 

I disliked the conflicts between members of the group. I feel that we all 
lost a little because of it. The lack of preparation on occasion bothered 
me. In social activities I felt the staff was forcing their norms on 
individuals and the group. 



...I feel veiry unsure of myself, my ability to communicate my thoughts and 
feelings as I intend them. These weeks have been one long frustrating 
confrontation. 

I wish I were more objective. I’m really confused. 

I do feel the members of my T-group — including staff — have a sincere regard 
for me and I appreciate that. So many personal Issues are unresolved. 

I frequently reject the data, but not the source, and so get nowhere — 
who's right? Maybe I'm out of touch with reality I 

I'Ve not lost confidence in my skills in the real world, but I wonder if 
I'll have the same problems there that I've had here and lose what self- 
esteem I nox</ cling to. Maudlin S 



...The program was most helpful to me in the area of Interpersonal relations 
with certain individuals. I have gained some insights to my weakness and 
strengths. The awareness of a weakness can be compensated for in many 
areas. Pat was seen most attentive when I was involved with a conflict 
with another staff member. I feel I have just begun to learn about Leader 
of Continuing Education training. I am looking forward to the follow- 
up seminars. Thank you very much. 



...Many times I find myself in a state of dissatisfaction about an activity. 
This Is not the case at this time. 

I appreciate the help the staff has given me, the support of good friends, 
and the enjoyment of their companionship. 

While not telling me what to do, the program helped me to find out how , 
once I knew within myself what I wanted to accomplish. 

I am probably more satisfied with life in general after this summer and I 
wish only to say, "Thank-you". This has been a moving experience for me 
and I will never forget my summer with the CERLI people at Charleston. 
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...I felt that I was not part of the group to a great extent, since my 
background was intellectually oriented » I felt that most of the 
participants were not scholarly people and I had to make great efforts 
to listen to some of the conversations. Many of them were types with 
whom I would not normally associate — so I had to make great efforts to 
adapt myself to the climate. I would like to see some type of outlet 
for various personality types. I felt that the value system of the 
group was being imposed upon me to a great extent. This I felt was 
dramatically opposed to the CERLI credo. In an effort to remove structure 
I felt that another was Imposed. 




















Appendix D; Participant Re-Evaluation Exercise 



Re-Evaluation Exercise On The 



CERLI Specialist of Continuing Education Training Program 
Charleston, Illinois July-August, 1968 



Instructions : 



During the weeks which have passed since your completion 
of the program, you have probably thought about the 
training program in which you participated. 

This form is designed to receive your opinions, judgments, 
and suggestions, now, about the training you received. 

Because this is not a test there are no right or wrong 
answers. We simply want to know your true feelings and 
reactions to your experiences during this program, now, 
so that future programs will get the benefit of your 
ideas and suggestions. 

Please answer every question, and if you wish, write 
additional comments next to the question or on the 
back of the page. 



THANK YOU VERY MUCH FOR YOUR COOPERATION 
















I 



a. 

b. 

c. 

d. 

e. 

f. 



g- 



h. 

i. 

j. 

k. 



1 . 



Indicate the degree of confidence you have in your abi lity to perform 
eac h of the, following by circling the appropriate number in each line ; 



1 

2 

3 

4 

5 



Extremely confident, completely sure of ability 
Highly confident, one or two weaknesses in ability 
Confident, good general ability with some weaknesses 
Mildly confident, many weaknesses in ability 
Not confident, completely unsure of ability 



Mean I 



to communicate ideas 
to understand new concepts 
to adapt to new situations 
to work well with others 
to organize others 
to view problems in a broad 
sense 

to subordinate self-interest 
for the sake of group interest 
to make decisions 
to understand others 
to make friends 
to be sympathetic to the 
problems of others 



1 

1 

5 

5 

1 



2 

2 

4 

4 

2 



3 

3 

3 

3 

3 



4 

4 

2 

2 



5 

5 

1 

1 

5 



2.5 
2.3 

2.6 
2.6 
2.5 



2.9 



1 

5 

5 

1 



2 

4 

4 

2 



3 

3 

3 

3 



4 

2 

2 

4 



5 

1 

1 

5 



2.9 

2.1 

2.9 

2.3 



3 



2.6 



^ satisfie d were you with the practice teaching components of the program ? 






completely 

satisfied 



78.0 

24.58 



completely 

dissatisfied 



3. How satisfied were you with the practice leader seminars? 



completely 



89.8 



dissatisfied 



completely 

satisfied 



16.94 



How satisfied were you with the staff critiques of the practice leader 
seminars? 



completely 

satisfied 



100.5 



17.15 



completely 

dissatisfied 




D-2 



iiiiiilii 







V 



5. How satisfied were you with the lecture and general sessions ? 



completely 

dissatisfied 

6. How satisfied were you with the search area ? 



73.5 
28.18 
completely 
satisfied 



8 . 



com 

sat 

How sat 


pletely compl 

isfied dissati 

Isfled were you with the evening small grouo sessions? 


23.32 

etely 

sfied 

1 85.10 


com 

dls 

How sat 






pletely 

satisfied 

Isfled were you with the formal and lnf< 


1 26.00 
completely 
satisfied 

Drmal social activities? 








94.60 



completely 

dissatisfied 



completely 

satisfied 



I_f CERLI conducts another leader training program of the same total 
length of time (l.e. 6 weeks), would you recommend that each listed 
activity be given about the same amount of time as this year, or that 
It be Increased or reduced In time ? 

(Please check one in each line, and remember that the total time is fixed) 



value* 


r recommend: 


More About same Less 

time as this year time 

+1 2 -3 Mean : 


a. Practice teaching 

b. Leader practice 
c« Leader seminars 

d. Staff critique of leader 

seminars 

e. General lecture sessions 

f. Search area period 

g. Evening groups 

h. Social mixing 
1. Free time 


2.3 


1.2 


1.6 


1.4 


2.5 


2.1 


2.2 


2.0 


1.8 
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10 . 



Please rate the coverage of substantive content in the 
program (circle one code in each column ) j 



li 

1 » too advanced 

2 a about right 

3 » too simple 



II; 

I * too general 

II a about right 

III a too detailed 



III; 

A » insufficient amount 
B a about right 
C a disproportionately 
large amount 



Topic; 




I; 




11; 




III; 




a. Goal establishment methodology 


1 


2 


3 


I II 


III 


A 


B 


C 


b. Data collection methodology 


2 


3 


1 


II III 


I 


B 


C 


A 


c. Resource methodology 


3 


1 


2 


m I 


II 


C 


A 


B 


d. Small group processes 


1 


2 


3 


I 11 


III 


A 


B 


C 


e. Confrontation techniques 


2 


3 


1 


II III 


I 


B 


C 


A 


f. Group functioning 


3 


1 


2 


III I 


II 


C 


A 


B 


g. Administrative methodology 


1 


2 


3 


I II 


III 


A 


B 


C 


"Mean” ratings; 


1; 




II; 


III; 










a. Goal establishment methodology 


2.1 




1.6 


1.6 










b. Data collection methodology 


2.4 




1.6 


1.3 










c. Resource methodology 


1.9 




1.7 


1.5 










d. Small group processes 


2.2 




1.6 


1.6 










e. Confrontation techniques 


2.2 




1.6 


1.6 










f. Group functioning 


2.2 




1.4 


1.4 










g. Administrative methodology 


2.1 




1.9 


1.9 











11* Please rate your overall satisfaction with the substantive content of the 
program . 



12 . 



extremely unsatisfactory, 
couldn't be worse 



82.50 



27.64 

extremely satisfactory, 
couldn't be better 



How much exchange of information and concepts — ^that is. cross fertilization 
of ideas — took place among your fellow members of the program ? 



minimum 

amount of exchange 
(practically none) 



80.80 



32.58 
maximum 
amount of exchange 
(completely overx<rhelming) 
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13. 




I; 



I 



I 

} 



f 



ink now of th e balance achieved by t he program in meeting voug spegjfic 
needs as compared to meeting t he needs of the group in gene ral « How 
^ o . u . ld you_judge the ba lance a chieved by the program in meeting fchese 



balance was excellent — 
couldn't have bean better 



82.50 

24.95 



balance was poor — 
couldn't have been worse 



What, in your opinion^ we re thre e important obieetlves of this CERLI program ? 
Sensitivity and Personality Chang e 



Ability to be honest with your self and others (.2) 

To develop in the participants a willingness to change 
To have participants work in an unstructured climate, thus being able to 
see that of chaos effective action frequently spx'ings 
i^^sight into how they (individuals) affect others 
Each individual develops at his own rate and to his own potential 

To try new behavior patterns without fear of failure— especially in a peer 
group ^ 

To be receptive to the needs and frustrations of fellow teachers 

Making the participants aware of our shortcomings in attitudinal behavior 
through sensitivity training 

To develop within the participants the skill to view objectively and with 

temperance the needs of the schools and especially teachers in the area of 
classroom behavior 



Promote Innovation via Teachers 



Promote new ideas in teacher education 

To get teachers to learn new classroom techniques 

Overall object of CERLI is to improve education by starting with teachers 

Train Le aders. Especially in Specific Skills 

To train people to become Specialists in Continuing Education by presenting 
them with the opportunity to practice the process of goal setting, con- 
frontation and evaluation 

Hopefully develop leaders for schools, who could provide leadership in 
bringing about behvavioral change (when needed) in teachers 

To train participants in the tactics of an effective SCE along the lines 
of group interaction, self-confrontation and innovation 

New Programs 

Develop a training package (3) 

To provide a self-perpetuating plan of group seminars for teacher improvement 



I 

I 

t: 
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Confrontation 



To make educators aware of their need for apathy of disadvantaged 
youngsters (sic) 

To get teachers to have a plan of self "assessment and evaluation, to take 
a closer (look) at their teaching objectives, student behavior and 
teaching behavior. (2) 



15.-26. 



The following questions refer to how adequately the program was 
administered by the staff. Circle the letter following each question 
to indicate rating of adequacy using the following code: 



A 

B 

C 

D 



extremely adequate; better than expected (value«l) 
adequate; about as expected (value*=2) 
adequate but less than expected (value*3) 
inadequate; much less than expected (value»4) 



Mean: 



15, 

16, 

17, 

18, 

19, 

20, 



21, 



22 . 



23. 

24. 

25. 

26. 



practice teaching arrangements 
beginning sessions on time 
carrying out sessions as scheduled 
changing schedule to meet group needs 
providing advance information on program changes 
maintaining objective relationships with 
group members 

helping group members with their special 
program needs 

helping group members with their special 
personal needs 

providing requested resources or facilities 
social mixing with group members 
resolving conflicts among group members 
other (specify) 



A 

D 

C 

B 

A 



B 

C 

A 

D 

B 



C 

B 

D 

A 

C 



D 

A 

B 

C 

D 



2.3 

2.5 

2.1 

1.7 

2.1 



B 



D 



A 



D 



B 



1.7 

1.9 



D 

A 

B 

C 

D 



C 

B 

D 

A 

C 



B 

C 

A 

D 

B 



A 

D 

C 

B 

A 



1.6 

1.7 

1.5 

2.2 



27. 



Space for additional comments about the administration of the program on 
the CERLl staff. 



The CERLl staff was very informal and receptive to the needs of the 

participants. However, some of the participants were unable to function 
properly within the group (staff and participant) norms. 

Too much time spent on "writing a program for Washington" and staff’s own 
internal problems the last two weeks of the institute. 

More work in the area of entry. 

Staff frequently playing it by ear with content materials. Sessions in- 
complete on occasion. 

Administration and staff too concerned with their social contacts of the 
evening before and made frequent references with cliches of which many 
participants were ignorant. 

General tone of comments among participants (on this) was complimentary. 
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2®* All in all, pleas e Indicate how you would rate the administration of the 
program , ~ 



I 92.60 

I 21.80 

extremely adequate, extremely Inadequate, 

better than expected much less than expected 

29. About how much of what you learned in this program will you be able to use 
when you begin functioning as an inservice leader in a school? 



none of it 



95.60 
26.57 
all of It 



30. looking back over the entire program, what specific part or thing was of 
greatest interest to you ? 



Sensitivity training (3) 

Leader seminars— putting cognitive input into action and it was then 
evaluated by the group or staff 

Opportunity to operate audio-visual equipment , especially video tape 
The insight in my own behavior, initiated in T-group but continued at 
Elks Club 

Confrontation with myself 

Recognizing need to set terminal goals in behavioral terms 
Establishing goals (measurable) as a leader and developing skills in 
confrontation and goal setting 

31 • What specific part or thing was of least interest to you ? 

Nightly social get-togethers at the local pub (to my knowledge) should have 
been more subtly publicized. Educators should at least be discreet enough 
not to publicize their secret sins to all (sic) 

Felt the social hour was imposed upon people 
T-groups 

General sessions 

Some general sessions vague and uninteresting for me — I was not involved 
Demonstration of special equipment and supplies used in teaching reading 
Filling out these forms 
Training program should be longer 

Seminar degenerated into rote exercise (our fault) real issues were 
avoided or at best treated surfacely (our fault) 



32., 33. (Values assigned to choices in the following questions indicated in 
parentheses) 

32. How ab out the total length of the program; do you feel it was too long , 
too short or about right ? 

■ J3) . too long (1) too short (2) a bout right / Mean; 2.0 • 
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^3* How has this program matched your expectations ? 

(1) b etter than expected 

(2) a bout what I expected 

(3) n ot as good as expected 

Mean ! 2.4 



Ke -rec ognize that some problems may arise in implementing and administering 
Jifls . ervice leader seminars. H ow much difficulty, if any, do you expect wit h 
■ea_ch_ of the f ollowing (circle one number in each item) ; 

1 = extreme difficulty 

2 = moderate difficulty 

3 = minor difficulty 

4 = little or no difficulty 



a. shortage of facilities (space, 

office supplies, etc.) 

b. shortage of funds 

c. pressure of regular duties 

d. lack of support from management or 

officials 

e. resistance to change by colleagues 

f. lack of authority 

g. Inadequate training 
other (specify) 

h. 

i. ~ 

i- I- 



12 3 
4 3 2 
12 3 

4 3 2 
12 3 
4 3 2 
12 3 

4 3 2 
12 3 
4 3 2 



4 

1 

4 

1 

4 

1 

4 

1 

4 

1 



Mean : 



2.3 

2.1 

2.7 

3.0 

2.5 

3.1 

2.6 



35. Idea lly , what follow-up activities do you expect from CERLI to help you 

. s.ucceed in implemen t ing and then administering an inservice leader 
program ? 



More T-group experiences (2) 

More information in the use of new Instruments (2) 

Almost implementation may be expected from CERLI due to the great amount 
of time that has expired since the Institute training unless there is an 
adequate review during our October conference 
Suggestions which are being, or have been, used with some degree of success 
by others (Exchange of ideas which are practical) 

Suggestions for group process exercises 

Definite instructions concerning definite activities which should be 
attempted 

Survey instruments to be prepared by CERLI for use in evaluating the program 

Probably won’t have any program in this area 

Three workshops 

List of materials and books 
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Support by listening to problems that may be encountered (Most valuable service 
simply offering the security of being available if help is needed) 

I feel self-sufficient in most areas 
Advise in handling particular problems 
Assistance as consultants, process observers 

Practice in designing and implementing entry and operational activities 




36 . Mark the line below at the point which best indicates the degree of your 
satisfaction with the program as a whole . 

« I 



I 



very unsatisfactory 



70.30 

25.58 

very satisfactory 



37 . Please write a short summary of your own personal reactions to this program . 

Mention anything, whether or not covered in the questions above, you liked 
best and also the things you disliked or thought of little importance . 



...What a difference a month makes! In retrospect I realize how valuable the 
program was to me, personally and professionally. I see now the reasons for 
my dissatisfactions and frustrations were caused by my own feelings of in- 
adequacy and unrealistic goals for myself and others. I rationalized and 
projected and fantasized my way through much of those six weeks! Now that I’m 
functioning in the field, I find much of the program applicable to my 
seminars. Bouquets to three staff members in particular: Russ, Ruth and Jan. 

Their support as staff and warmth as people were of great help to me. 



...Very definite negative reactions concerning any value at all received from 
T-group sessions. Seminars involving confrontation and goal setting could 
have been of much greater permanent value if the various procedures utilized 
by different members of the group could have been lifted from the tape 
recordings, cataloged, and those considered to be most successful, utilized 
as models for demonstration purposes for the other participants. 




i k 



3 



,1 



M 



1 



% 



I 



'i 



...Liked the program because it wasn't what I had expected. Really liked the 
informality of the staff and its encouragement of new behavior from the 
participants. The part-time staff members were not however too helpful and 
they did not really contribute to my favorable opinion of the session. 

Wish my T-group session had been more effective. Enjoyed the seminars be- 
cause I was forced to consider the needs of each member and I had to use 
tact to get the results that I wanted. This particular experience has been 
a great deal of help to me already. 
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•*«The classroom teaching and following seminar were really basic activities 
In understanding the purpose of the program. To see oneself objectively Is 
always relevant for self-improvement. I enjoyed the T-groups because here 
one could feel free to pour out our feelings under conditions where there would 
be a friendly and sympathetic atmosphere. 



I ... Carry— over Into the regular school program week. Ideas of program highly 

I acceptable but most teachers shy away — reasons for not participating 

I numerous, but most frequent are time and credit. CERLI must have more say 

I In Its selection of participants. 

I 

f — . . 

I ...I felt the program did a lot for me personally. I gained a considerable 

i sniount of self-confidence In dealing with people. I am far more receptive 

[ to change. I felt that there was a need for more Input on how to deal with 

I a seminar (things to look for and ways to work them out.) Times I felt that 

I group tended to destroy Individualism. Another problem was the fact 

I that several Individuals were commuting and consequently they had other 

I Interests. 



C 

j ■ ™ ** ■ — ^ ..11 ■ ^ 

...It Is extremely Interesting to observe the similarity of direction In plans 
for Improvement In education being taken by educators all over the country, 
the most Important of which may be community Involvement. My first knowledge 
of such programs was Introduced when I perused "The Human Relations School" 
by Fox & Llppltt In the search area last summer. This reading paved the way 
for a recent conference of The Research Council of The Great Cities Program 
for School Improvement. Here I was made aware that community advisory 
: councils may be the answer to Improved education. Such Innovations, combined 

sensitivity training Initiated In the school hierarchy from the superin- 
tendents on down could be the dawn of a new day In the field of education. 



...The summer Institute was most helpful to me In many ways. It was the first 
time I had an experience of working and living with people for a period of 
six weeks. I gained Insight Into ny own feeling of a "locked In" experience. 

I had many ambiguous feeling (sic) toward some staff members and their 
ability to cope highly emotional experiences (sic). I enjoyed living In 
the dorm with the students, this gave me an opportunity to test some of my 
own feeling. I developed a strong relationship with several of the students 
(girls). I had a few depressing experiences, but when I look back the summer 
was one of the most valuable experiences I have had. 
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•••Without being a smart~ass, I would like to say that the many Instruments 
have, at times, been trying^ The program has been personally rewarding 
and extremely useful In my work with teachers • I hope the follow-up 
session Is as rewarding^ 



Please name the three participants In the program this summer whom you feel 
will be the most successful SCEs and why » 

!• 

2 . 

3 ^ 

Please name the three whom you feel will be least successful and why , 

!• 

1 . 

3 ^ - 

PLEASE WRITE YOUR CODE NUMBER HERE 



I 

I Assessor* s Note ; 

I Though participants did Indicate reasons why they predicted either successful or 
I unsuccessful performance of their fellow-participants In an actual situation, we 

I present only the tabulation of responses • Contextually , the statements convey 

I judgments and rationale that the participant's personality, background, experience 
I at the Institute and other variables somewhat skew or obscure • The fact that 

participants i^^ll,i^^7,i^fl4 and M were cited In both questions 38 and 39 Illustrates 
I the mercurial nature of the "why's" explicating statements about predicted success • 




I 38 • Most successful SCEs: 



I Participant: Total nominations: 










39 . Least successful SCEs : 



Participant : 



Total nominations ; 



16 7 

15 6 

6 4 

2 3 

11 2 

7 1 

14 1 

4 1 

9 1 



D-12 









Appendix E: Staff and Participants* Perceptions of Program* s Success 



(See "Conclusion", page 19, for detailed information about this 
assessment. Errata: the staff did not answer #12 rather than 

#13 as stated in the texto) 



CODE: 



S = Staff Evaluation Exercise (Appendix B) 

P-R =* Participant Re-* Evaluation Exercise (Appendix C) 
P - Participant Evaluation Exercise (Appendix D) 



Scale - 126mm 



1 • How satisfied were you with the teaching components of the program ? 



S 87o60 
P-R - 78.00 
P = 63.20 






pd 

T 






completely 

satisfied 



Q 



completely 

dissatisfied 



2 . How satisfied were you with 'the practice leader seminars ? 



S = 88.20 
P-R 89o80 

P =93.1 
0— > 



I 



completely 

dissatisfied 



satisfied 



3 . How satisfied were you with the staff critiques of the practice 



leader seminars? 



S = 104.00 
P-R = 100.50 



P = 83.30 ^ 



c/3 



P4 



completely 

satisfied 



^™”0 



dissatisfied 



E“1 































4. How satisfied were you with the lecture and Reneral sessions ' 



S = 93.80 
P-R = 73.20 
P * 67.90 
0- 



p^i 

I 

P4 PL* 



completely 

dissatisfied 



CO 



completely 

satisfied 



5 . How satisfied were you with the search area ? 



S * 108.80 
P-R =* 85.60 
P « 93.30 

CO 



completely 

satisfied 



1: 



(Xi 

I 



<-0 



completely 

dissatisfied 



6. How satisfied were you with the evening small group sessions ? 



S * 108.80 
P-R = 85.60 
P =93.30 
0 * 



pic* 

I 



CO 



completely 

dissatisfied 



completely 

satisfied 



7. How satisfied were you with the formal and informal social activities ? 



S = 102.00 
P-R = 99.60 



P =^90.60 

) — ^ ru 


1 















completely 
dissatisfied 



completely 

satisfied 
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Please rate your o verall satisfaction with the substantive 
content of the program . 



S « 91.20 
P-R = 82.50 
P « 80.40 




0 ^ 



extremely unsatisfactory, 
couldn't be worse 



extremely satisfactory, 
couldn't be better 



^ much exchang e of Information and concepts— that is cross— 

Qf ideas— took place among your fellow members of 

the sroc^ram ? 

(The term fellow members may seem ambiguous for it does not 
specify participants or staff or both.) 

S « 102.00 
P-R = 80.60 
P *J3.60 

0 — ^ I 



specific needs a s compared to meeting the needs of the group in 

general. How wo uld you judge the balance achieved by the program 

in meeting these different needs ? 

(The term group does not clearly identify its members as 
participants or staff or both.) 

S « 98.40 
P-R = 81.60 
P « 83.40 

I 






minimum 

amount of exchange 
(practically none) 



maximum 

amount of exchange 
(completely overwhelming) 



* Think now of the balance achieved by the program in meeting your 




0 



balance was excellent- 
couldn't have been better 



balance was poor- 
couldn't have been worse 



All In all« please indicate how you would rate the administration 
of the program * 



S * 112.20 
P-R » 92.20 
P « 87.50 

CO 



p4 

I 

fH 



extremely adequate, 
better than expected 



0 



extremely inadequate, 
much less than expected 



12* About how much of what you learned in this program will you be able 

to use when you begin functioning as an inservico leader in a school? 



P-R « 95.6 
P * 98.10 



none of it all of it 



Pi 

I 



13. Merk the line below at the point which best indicates the degree 
of your satisfaction with the program as a whole . 



S - 103.00 

P-R « 89.00 

P -92.60 
0 — > 



very unsatisfactory 



Pi 

T Pm CO 
Pi 



very satisfactory 
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